Oct 30 18:43:31.698582 ip-10-0-143-248 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 18:43:31.698592 ip-10-0-143-248 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 18:43:31.698599 ip-10-0-143-248 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 18:43:31.698819 ip-10-0-143-248 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 18:43:41.740732 ip-10-0-143-248 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 18:43:41.740747 ip-10-0-143-248 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 30cd5602b9c7447186186bd3c678ce46 --
Oct 30 18:45:32.245524 ip-10-0-143-248 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:45:32.764674 ip-10-0-143-248 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:45:32.764674 ip-10-0-143-248 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:45:32.764674 ip-10-0-143-248 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:45:32.764674 ip-10-0-143-248 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:45:32.772297 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772198    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:32.772297 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772296    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:32.772297 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772301    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772304    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772307    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772310    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772313    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772315    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772318    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772322    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772324    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772327    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772330    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772332    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772335    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772337    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772340    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772343    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772345    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772348    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772350    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:32.772395 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772353    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772356    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772359    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772361    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772364    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772367    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772377    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772380    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772383    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772386    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772388    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772391    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772393    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772396    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772399    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772402    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772406    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772410    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772413    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:32.772833 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772416    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772418    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772421    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772424    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772426    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772429    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772432    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772436    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772440    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772443    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772446    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772448    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772451    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772453    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772456    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772459    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772462    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772465    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772467    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:32.773288 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772470    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772473    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772476    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772479    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772481    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772484    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772486    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772489    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772491    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772494    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772497    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772500    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772502    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772505    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772508    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772510    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772512    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772515    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772517    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772520    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:32.773746 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772522    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772525    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772527    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772530    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772533    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772535    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772538    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772541    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772544    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772547    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772550    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772552    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772555    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772557    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772560    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772563    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772565    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772568    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772571    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772573    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:32.774219 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772576    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772578    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772581    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772583    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772586    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772590    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.772593    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773421    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773428    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773432    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773435    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773438    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773441    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773444    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773446    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773449    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773452    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773454    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773457    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:32.774730 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773460    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773463    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773466    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773468    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773471    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773474    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773477    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773480    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773483    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773486    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773489    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773491    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773494    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773498    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773502    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773506    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773509    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773511    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773514    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773517    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:32.775183 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773519    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773522    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773524    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773527    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773530    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773533    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773535    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773538    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773541    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773543    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773546    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773548    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773551    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773554    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773557    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773560    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773562    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773565    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773568    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773570    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:32.775673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773573    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773581    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773584    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773587    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773589    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773592    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773594    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773597    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773600    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773602    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773605    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773607    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773609    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773612    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773615    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773617    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773620    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773623    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773626    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773628    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:32.776148 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773631    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773633    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773636    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773638    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773641    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773645    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773649    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773652    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773655    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773658    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773660    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773662    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773665    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773667    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773671    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773673    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773676    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773678    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773681    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:32.776636 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773683    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773686    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773688    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773691    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773694    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773697    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773700    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773702    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773705    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773707    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773710    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773712    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773715    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773718    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.773720    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774223    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774229    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774232    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774235    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774238    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:32.777081 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774255    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774258    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774261    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774264    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774266    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774269    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774272    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774275    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774278    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774281    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774283    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774286    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774289    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774292    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774294    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774297    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774299    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774302    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774305    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:32.777552 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774307    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774310    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774313    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774315    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774318    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774320    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774326    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774329    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774334    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774338    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774341    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774344    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774347    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774349    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774352    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774354    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774357    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774360    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774363    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774365    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:32.778016 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774368    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774371    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774373    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774376    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774379    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774382    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774384    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774387    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774389    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774392    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774394    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774397    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774399    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774402    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774405    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774407    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774410    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774412    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774416    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774419    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:32.778540 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774422    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774424    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774427    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774429    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774432    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774434    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774437    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774439    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774442    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774444    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774447    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774450    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774453    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774455    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774458    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774461    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774464    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774466    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774469    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:32.779002 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774474    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774478    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774481    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774484    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774487    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774489    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774491    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774494    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774496    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774499    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774501    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774505    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774508    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774511    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774515    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774517    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774520    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774522    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774525    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774527    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:32.779520 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774530    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774532    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.774534    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774638    2577 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774645    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774652    2577 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774657    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774662    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774671    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774678    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774683    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774686    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774689    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774693    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774697    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774700    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774703    2577 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774706    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774709    2577 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774712    2577 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774715    2577 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774721    2577 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774724    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774729    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:45:32.780035 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774733    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774736    2577 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774741    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774744    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774747    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774751    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774754    2577 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774757    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774760    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774763    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774766    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774772    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774775    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774778    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774781    2577 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774784    2577 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774787    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774799    2577 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774802    2577 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774805    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774808    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774811    2577 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774816    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774819    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774822    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:45:32.780640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774825    2577 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774828    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774831    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774834    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774837    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774840    2577 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774844    2577 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774847    2577 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774851    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774854    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774858    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774861    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774864    2577 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774867    2577 flags.go:64] FLAG: --help="false"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774870    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774873    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774876    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774879    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774884    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774887    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774890    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774893    2577 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774896    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:45:32.781293 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774899    2577 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774902    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774910    2577 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774913    2577 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774916    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774919    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774922    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774925    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774928    2577 flags.go:64] FLAG: --lock-file=""
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774931    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774933    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774936    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774942    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774945    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774948    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774952    2577 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774955    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774959    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774961    2577 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774964    2577 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774969    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774972    2577 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774975    2577 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774979    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774982    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:45:32.781841 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774985    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774988    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774992    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774995    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.774998    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775013    2577 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775016    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775019    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775022    2577 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775025    2577 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775033    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775037    2577 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775039    2577 flags.go:64] FLAG: --port="10250"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775042    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775045    2577 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0eb2697fdb173e241"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775048    2577 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775052    2577 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775055    2577 flags.go:64] FLAG: --register-node="true"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775059    2577 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775063    2577 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775065    2577 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775068    2577 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775072    2577 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775076    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775079    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775082    2577 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:45:32.783077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775085    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775088    2577 flags.go:64] FLAG: --runonce="false"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775091    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775094    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775097    2577 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775100    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775103    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775106    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775110    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775113    2577 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775116    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775119    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775122    2577 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775125    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775128    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775131    2577 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775133    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775139    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775142    2577 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775145    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775148    2577 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775151    2577 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775154    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775157    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775160    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:45:32.783730 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775163    2577 flags.go:64] FLAG: --v="2"
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775167    2577 flags.go:64] FLAG: --version="false"
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775171    2577 flags.go:64] FLAG: --vmodule=""
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775176    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775180    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775354    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775359    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775362    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775364    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775367    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775370    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775372    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775375    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775377    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775380    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775384    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775387    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775389    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775392    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775394    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:32.784324 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775397    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775399    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775402    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775404    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775407    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775415    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775419    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775423    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775426    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775428    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775431    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775433    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775436    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775439    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775442    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775446    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775449    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775451    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775454    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775456    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:32.784796 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775459    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775461    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775464    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775467    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775469    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775472    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775474    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775478    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775481    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775483    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775486    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775489    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775491    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775494    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775496    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775499    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775502    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775504    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775508    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775510    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:32.785279 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775513    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775516    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775518    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775521    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775523    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775526    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775528    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775532    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775535    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775537    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775540    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775542    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775545    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775547    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775550    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775553    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775555    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775558    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775560    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:32.785958 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775564    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775567    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775569    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775571    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775574    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775577    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775581    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775584    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775587    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775590    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775592    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775595    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775598    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775600    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775603    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775605    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775608    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775625    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775630    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775633    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:32.786787 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775637    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775640    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775642    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775645    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775648    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775650    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775653    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775656    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775658    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775661    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775663    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775666    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.775694    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775737    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775742    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775745    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:32.787300 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775748    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775751    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775754    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775756    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775759    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775761    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775764    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775766    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775769    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775772    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775775    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775777    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775780    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775782    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775785    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775788    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775790    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775793    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775796    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775798    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:32.787673 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775801    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775804    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775806    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775809    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775811    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775813    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775816    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775819    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775823    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775825    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775827    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775830    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775832    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775835    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775837    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775840    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775842    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775844    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775847    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:32.788134 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775850    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775852    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775854    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775858    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775861    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775864    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775866    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775869    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775871    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775874    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775876    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775881    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775884    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775887    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775889    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775892    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775894    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775897    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775899    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:32.788622 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775902    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775904    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775909    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775911    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775914    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775916    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775919    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775921    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775924    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775926    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775929    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775932    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775934    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775937    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775939    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775942    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775944    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775947    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775950    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:32.789070 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775952    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775955    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775957    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775960    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775962    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775965    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775967    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775970    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775972    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775975    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775977    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775980    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775982    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775985    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775987    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775991    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775994    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775996    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.775999    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:32.789527 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.776003    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.776007    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.776011    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.776014    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.776017    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.776020    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.776022    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.783963    2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.783980    2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.785466    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.785542    2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785791    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785808    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785813    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785818    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785824    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785830    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785835    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785839    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:32.789975 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785843    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785848    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785858    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785863    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785867    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785872    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785876    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785881    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785887    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785891    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785896    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785901    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785905    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785910    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785916    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785925    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785929    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785933    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785939    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785944    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:32.790446 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785948    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785952    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785957    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785962    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785967    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785972    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785977    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785981    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785990    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.785996    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786000    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786005    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786009    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786014    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786019    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786023    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786028    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786032    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786036    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:32.790905 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786040    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786045    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786054    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786058    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786062    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786067    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786071    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786076    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786082    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786088    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786093    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786098    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786102    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786106    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786116    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786121    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786126    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786130    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786135    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:32.791482 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786140    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786144    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786148    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786153    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786158    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786162    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786166    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786170    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786179    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786183    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786187    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786191    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786195    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786200    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786204    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786209    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786213    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786217    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786222    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786226    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:32.791925 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786235    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786239    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786260    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786265    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786269    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786273    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786277    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786282    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786287    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786291    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786296    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786301    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786305    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786314    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786318    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786323    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786326    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786331    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786338    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:32.792414 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786345    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.786405    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786916    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786930    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786935    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786938    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786941    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786944    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786947    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786949    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786952    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786955    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786958    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786960    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786963    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786965    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:32.792871 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786968    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786971    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786973    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786976    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786979    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786982    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786985    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786987    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786990    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786992    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786995    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.786998    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787000    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787003    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787006    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787010    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787012    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787015    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787018    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:32.793263 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787020    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787023    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787026    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787028    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787031    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787033    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787037    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787039    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787042    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787044    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787047    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787050    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787053    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787055    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787058    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787061    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787063    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787066    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787068    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787071    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:32.793703 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787074    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787076    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787079    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787081    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787084    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787087    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787090    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787092    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787096    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787099    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787101    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787104    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787107    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787109    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787112    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787115    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787117    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787119    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787122    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787124    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:32.794188 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787127    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787130    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787133    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787136    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787138    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787141    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787144    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787147    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787149    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787152    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787154    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787157    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787160    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787163    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787165    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787169    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787173    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787177    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787180    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787182    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:32.794717 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787185    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787187    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787190    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787192    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787195    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787199    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787203    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787206    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787209    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787213    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787215    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787218    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787220    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.787275    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787313    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:32.795205 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787320    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787323    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787326    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787329    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787332    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787335    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787337    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787340    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787343    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787347    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787349    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787352    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787355    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787357    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787360    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787362    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787365    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787368    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787370    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787373    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:32.795719 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787376    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787378    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787381    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787383    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787386    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787388    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787391    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787393    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787397    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787402    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787405    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787408    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787415    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787418    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787421    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787424    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787427    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787429    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787433    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:32.796276 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787437    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787441    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787445    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787448    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787450    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787453    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787456    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787458    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787461    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787464    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787466    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787469    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787471    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787474    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787476    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787479    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787481    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787485    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787487    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787490    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:32.796742 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787492    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787495    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787498    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787500    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787503    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787506    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787510    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787512    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787515    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787517    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787520    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787523    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787525    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787528    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787532    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787534    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787537    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787539    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787542    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:32.797221 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787545    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787547    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787550    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787553    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787555    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787558    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787561    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787563    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787566    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787568    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787571    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787574    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787576    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787579    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787581    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787584    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787587    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787589    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787594    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787620    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:32.797683 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787626    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787630    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787632    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787636    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787639    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787642    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:32.787644    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.787709    2577 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.790009    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.791235    2577 server.go:1014] "Starting client certificate rotation"
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.791347    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.791386    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.794706    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:45:32.798185 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.795827    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:45:32.810585 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.810569    2577 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:45:32.816640 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.816625    2577 log.go:25] "Validated CRI v1 image API"
Oct 30 18:45:32.817496 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.817470    2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:45:32.820655 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.820627    2577 fs.go:135] Filesystem UUIDs: map[6e667742-f335-46c7-b2a5-ab43ac269d5b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d25e609b-5ef7-4b07-bc51-17e2a17f381b:/dev/nvme0n1p4]
Oct 30 18:45:32.820729 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.820654    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 18:45:32.828169 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.828052    2577 manager.go:217] Machine: {Timestamp:2026-10-30 18:45:32.825300948 +0000 UTC m=+0.433455522 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100094 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d951ae95e3b218cc4c62743634040 SystemUUID:ec2d951a-e95e-3b21-8cc4-c62743634040 BootID:30cd5602-b9c7-4471-8618-6bd3c678ce46 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:e6:8a:8d:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:e6:8a:8d:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:e2:0b:e1:36:b0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:45:32.829015 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.829003    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:45:32.829136 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.829121    2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:45:32.832545 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.832519    2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:45:32.832684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.832547    2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-248.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:45:32.833334 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.833324    2577 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:45:32.833371 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.833335    2577 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:45:32.833371 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.833363    2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:45:32.833426 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.833377    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:45:32.834425 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.834415    2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:45:32.838450 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.838435    2577 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:45:32.838571 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.838562    2577 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:45:32.839514 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.839504    2577 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:45:32.839558 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.839517    2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:45:32.839558 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.839528    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:45:32.839558 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.839537    2577 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:45:32.839679 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.839559    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:45:32.840456 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.840442    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:45:32.844530 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.844509    2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:45:32.851744 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.851731    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:45:32.853337 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.853325    2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:45:32.853383 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.853355    2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:45:32.853509 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.853498    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:45:32.853546 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.853513    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:45:32.853546 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.853520    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:45:32.853546 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.853526    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:45:32.853546 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.853532    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:45:32.853546 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.853538    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:45:32.854887 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.854878    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:45:32.854960 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.854887    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:45:32.854960 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.854894    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:45:32.854960 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.854900    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:45:32.854960 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.854923    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:45:32.854960 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.854933    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:45:32.856096 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.856084    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:45:32.856137 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.856099    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:45:32.860019 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.860005    2577 server.go:1263] "Started kubelet"
Oct 30 18:45:32.860152 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.860094    2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:45:32.860272 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.860134    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:45:32.860272 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.860213    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:45:32.860712 ip-10-0-143-248 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:45:32.862853 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.862834    2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:45:32.862989 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.862964    2577 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:45:32.867095 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:32.867076    2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 18:45:32.868983 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.868969    2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:45:32.869043 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.868988    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:45:32.869092 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:32.869056    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:32.869731 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.869714    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:45:32.869731 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.869714    2577 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:45:32.869869 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.869739    2577 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:45:32.869869 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.869812    2577 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:45:32.869869 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.869819    2577 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:45:32.870009 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.869884    2577 factory.go:55] Registering systemd factory
Oct 30 18:45:32.870009 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.869901    2577 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:45:32.870180 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.870165    2577 factory.go:153] Registering CRI-O factory
Oct 30 18:45:32.870223 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.870184    2577 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:45:32.870301 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.870257    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:45:32.870301 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.870283    2577 factory.go:103] Registering Raw factory
Oct 30 18:45:32.870301 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.870298    2577 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:45:32.871337 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.871319    2577 manager.go:319] Starting recovery of all containers
Oct 30 18:45:32.882467 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.882441    2577 manager.go:324] Recovery completed
Oct 30 18:45:32.887714 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.887700    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:32.889538 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.889507    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:32.889609 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.889556    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:32.889609 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.889568    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:32.890119 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.890105    2577 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:45:32.890119 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.890116    2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:45:32.890253 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.890136    2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:45:32.890253 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.890148    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:32.890806 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.890792    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:32.890857 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.890817    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:32.890857 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.890829    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:32.891498 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.891487    2577 policy_none.go:50] "Start"
Oct 30 18:45:32.891538 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.891503    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:45:32.891538 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.891516    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:45:32.892856 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.892843    2577 policy_none.go:44] "Start"
Oct 30 18:45:32.924886 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.924848    2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.930448    2577 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:32.931139    2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.931154    2577 server.go:87] "Starting device plugin registration server"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.931401    2577 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.931410    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.931514    2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.931603    2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.931610    2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:32.932141    2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:32.932206    2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.942603    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.943858    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.943876    2577 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.943895    2577 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:45:32.945553 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:32.943925    2577 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 18:45:32.994426 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:32.994333    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-248.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 18:45:32.994426 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:32.994347    2577 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-248.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 18:45:33.031780 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.031759    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:33.032736 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.032719    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:33.032811 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.032762    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:33.032811 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.032780    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:33.032811 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.032807    2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:33.040728 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:33.039518    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-248.us-west-2.compute.internal.18e36301114d2d7d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-248.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-248.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:45:32.859977085 +0000 UTC m=+0.468131656,LastTimestamp:2026-10-30 18:45:32.859977085 +0000 UTC m=+0.468131656,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-248.us-west-2.compute.internal,}"
Oct 30 18:45:33.044051 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.044033    2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"]
Oct 30 18:45:33.044100 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.044089    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:33.044960 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.044943    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:33.045030 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.044972    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:33.045030 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.044984    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:33.046913 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.046901    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:33.047078 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.047066    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:33.047113 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.047093    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:33.047705 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.047690    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:33.047705 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.047697    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:33.047796 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.047718    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:33.047796 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.047719    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:33.047796 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.047728    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:33.047796 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.047733    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:33.049212 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.049194    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:33.049308 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.049222    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:33.050291 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.050270    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:33.050391 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.050333    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:33.050391 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.050350    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:33.060356 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:33.060323    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 18:45:33.060469 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:33.060445    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Oct 30 18:45:33.060805 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:33.060786    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-248.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 18:45:33.065663 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:33.065634    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:33.068560 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:33.068540    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:33.075275 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.075237    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:45:33.075374 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:33.075302    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gc8dr"
Oct 30 18:45:34.875177 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.875144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.875349 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.875178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.875349 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.875208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bf5677d5ce41c447fb39800129b147-config\") pod \"kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"d0bf5677d5ce41c447fb39800129b147\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.944425 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.944393    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:34.948043 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.948025    2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.948107 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:34.948051    2577 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-248.us-west-2.compute.internal\": node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:34.975483 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.975409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.975483 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.975442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.975483 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.975464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bf5677d5ce41c447fb39800129b147-config\") pod \"kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"d0bf5677d5ce41c447fb39800129b147\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.975662 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.975498    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.975662 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.975511    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:34.975662 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:34.975498    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bf5677d5ce41c447fb39800129b147-config\") pod \"kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"d0bf5677d5ce41c447fb39800129b147\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:35.072048 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.072013    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.073878 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.073861    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:35.079496 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.079482    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:35.172166 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.172128    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.272992 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.272902    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.373203 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.373161    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.474003 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.473956    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.495720 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.495691    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 18:45:35.495889 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.495869    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:45:35.495969 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.495887    2577 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a08f964132a324d829023bb7d5fff707-ee203c489136d9c1.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-143-248.us-west-2.compute.internal.18e363011310412c\": read tcp 10.0.143.248:56646->18.222.87.210:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-143-248.us-west-2.compute.internal.18e363011310412c  default   5090 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-248.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-248.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-248.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:45:32 +0000 UTC,LastTimestamp:2026-10-30 18:45:33.044957725 +0000 UTC m=+0.653112298,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-248.us-west-2.compute.internal,}"
Oct 30 18:45:35.574339 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.574276    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.674867 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.674836    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.700769 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.700751    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:35.701675 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.701660    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:35.701739 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.701687    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:35.701739 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.701698    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:35.775387 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.775359    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.789653 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.789629    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:40:34 +0000 UTC" deadline="2028-07-23 17:02:14.712972394 +0000 UTC"
Oct 30 18:45:35.789653 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:35.789650    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15166h16m38.923325615s"
Oct 30 18:45:35.875724 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.875660    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:35.976475 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:35.976439    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:36.025222 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.025199    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:36.076571 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.076543    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:45:36.141547 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.141481    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:36.159178 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.159154    2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:36.174163 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.174136    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:36.180443 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:36.180404    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58894da434aca988e2149b0960c93e80.slice/crio-d417883d4c4541976655c44ca1b09639ea9e465b723673cd5b7da7596a6c76d5 WatchSource:0}: Error finding container d417883d4c4541976655c44ca1b09639ea9e465b723673cd5b7da7596a6c76d5: Status 404 returned error can't find the container with id d417883d4c4541976655c44ca1b09639ea9e465b723673cd5b7da7596a6c76d5
Oct 30 18:45:36.180844 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:36.180825    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0bf5677d5ce41c447fb39800129b147.slice/crio-fb81cc9b032d077ea698198cdb7a5af6674499023cce89e4cecdddc1341b2886 WatchSource:0}: Error finding container fb81cc9b032d077ea698198cdb7a5af6674499023cce89e4cecdddc1341b2886: Status 404 returned error can't find the container with id fb81cc9b032d077ea698198cdb7a5af6674499023cce89e4cecdddc1341b2886
Oct 30 18:45:36.186824 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.186800    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:45:36.232340 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.232314    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:45:36.233402 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.233388    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:45:36.286143 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.286119    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:45:36.546141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.546059    2577 apiserver.go:52] "Watching apiserver"
Oct 30 18:45:36.575282 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.575245    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 18:45:36.653535 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.653483    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"58894da434aca988e2149b0960c93e80","Type":"ContainerStarted","Data":"d417883d4c4541976655c44ca1b09639ea9e465b723673cd5b7da7596a6c76d5"}
Oct 30 18:45:36.654444 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.654423    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"d0bf5677d5ce41c447fb39800129b147","Type":"ContainerStarted","Data":"fb81cc9b032d077ea698198cdb7a5af6674499023cce89e4cecdddc1341b2886"}
Oct 30 18:45:36.736038 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.736009    2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:45:36.736365 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.736348    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-phfz4","openshift-multus/multus-additional-cni-plugins-ns79r","kube-system/global-pull-secret-syncer-kml2f","kube-system/konnectivity-agent-xhv7g","openshift-multus/multus-9jtw5","openshift-multus/network-metrics-daemon-29t9c","openshift-network-diagnostics/network-check-target-f8whs","openshift-ovn-kubernetes/ovnkube-node-dchr9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd","openshift-dns/node-resolver-6rdvd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal","openshift-network-operator/iptables-alerter-sg2mz","kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-t75nr"]
Oct 30 18:45:36.738918 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.738902    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:36.738999 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.738980    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:36.740380 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.740368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.741893 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.741874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:36.743205 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.743183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.744512 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.744495    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.745962 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.745946    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.747296 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.747279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.748850 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.748832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.750625 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.750603    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.750710 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.750671    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:36.751968 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.751953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:36.752038 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.752001    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:36.753435 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.753417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.754920 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.754904    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.775587 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.775569    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:45:36.781877 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.781855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-cni-binary-copy\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.781959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.781882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-conf-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.781959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.781904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-cnibin\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.781959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.781925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.781959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.781942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.781959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.781957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-bin\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.781972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20572ffb-cc67-4066-b50c-22489eeed6e7-ovn-node-metrics-cert\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.781991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.782141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-var-lib-kubelet\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.782141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:36.782141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-cnibin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.782141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782104    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.782141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-kubelet-config\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.782141 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kppzz\" (UniqueName: \"kubernetes.io/projected/b4075d5d-2d2d-4e0e-b913-12715719e711-kube-api-access-kppzz\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-env-overrides\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-kubernetes\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-host-slash\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-dbus\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782279    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-hosts-file\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-tmp-dir\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh6dp\" (UniqueName: \"kubernetes.io/projected/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-kube-api-access-vh6dp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-systemd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.782458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-socket-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-registration-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782515    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-sys-fs\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-modprobe-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-kubelet\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-var-lib-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-config\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-device-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-konnectivity-ca\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-system-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-lib-modules\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-k8s-cni-cncf-io\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782788    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-netd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.782897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782950    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-script-lib\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.782983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gmll\" (UniqueName: \"kubernetes.io/projected/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kube-api-access-2gmll\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysconfig\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-agent-certs\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m82st\" (UniqueName: \"kubernetes.io/projected/20a81c77-50d1-4e5b-a59d-43d455fb842d-kube-api-access-m82st\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4075d5d-2d2d-4e0e-b913-12715719e711-host\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-systemd-units\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-host\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-kubelet\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-multus-daemon-config\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-system-cni-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-sys\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-netns\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-node-log\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783395 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783349    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-run\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhcnv\" (UniqueName: \"kubernetes.io/projected/224cd297-364e-466b-a43e-3ac6a2de7d29-kube-api-access-bhcnv\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-bin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-multus\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-etc-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-ovn\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783466    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq9pz\" (UniqueName: \"kubernetes.io/projected/20572ffb-cc67-4066-b50c-22489eeed6e7-kube-api-access-fq9pz\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783484    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-systemd\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-tuned\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-socket-dir-parent\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-hostroot\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4b4m\" (UniqueName: \"kubernetes.io/projected/b202127e-afab-4cac-bee8-e93533bc25d5-kube-api-access-n4b4m\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwnlq\" (UniqueName: \"kubernetes.io/projected/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-kube-api-access-zwnlq\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-slash\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-netns\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-log-socket\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.783890 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.784381 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-multus-certs\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.784381 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-etc-kubernetes\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.784381 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-tmp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.784381 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-iptables-alerter-script\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.784381 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4075d5d-2d2d-4e0e-b913-12715719e711-serviceca\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.784381 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-conf\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.784381 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.783925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m4sr\" (UniqueName: \"kubernetes.io/projected/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-kube-api-access-8m4sr\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.790617 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.790584    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:45:36.790916 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.790868    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:45:36.790916 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.790877    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:45:36.790916 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.790901    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:45:36.790916 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.790908    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:45:36.791102 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.790921    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:45:36.791102 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.790909    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-29qlm\""
Oct 30 18:45:36.793905 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.793889    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:45:36.793988 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.793925    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-56wtl\""
Oct 30 18:45:36.794033 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.794001    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:45:36.794077 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.794064    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:45:36.794156 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.794143    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:45:36.794197 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.794145    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:45:36.794197 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.794163    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:45:36.794263 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.794235    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-r47g8\""
Oct 30 18:45:36.794263 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.794255    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:45:36.831915 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.831864    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:45:36.832031 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.832016    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lj6ts\""
Oct 30 18:45:36.836923 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.836907    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hz6tx\""
Oct 30 18:45:36.836989 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.836908    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5tsjg\""
Oct 30 18:45:36.836989 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.836911    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n4vwp\""
Oct 30 18:45:36.836989 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.836948    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bpjwm\""
Oct 30 18:45:36.837176 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837159    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-77vpl\""
Oct 30 18:45:36.837241 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837229    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:45:36.837289 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837254    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:45:36.837338 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837289    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:45:36.837338 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837302    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:45:36.837432 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837350    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:45:36.837432 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837357    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:45:36.837432 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837370    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:45:36.837432 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837372    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:45:36.837432 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837382    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:45:36.837432 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837380    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:45:36.837432 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.837401    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:45:36.840007 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.839994    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:45:36.840433 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.840422    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:45:36.840505 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.840493    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:45:36.840538 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.840527    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mf8nx"
Oct 30 18:45:36.884244 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-host\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884357 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-kubelet\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884357 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-multus-daemon-config\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884357 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-system-cni-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.884357 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884335    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-system-cni-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.884357 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884347    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-host\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884347    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-kubelet\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-sys\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-netns\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-node-log\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884471    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-netns\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884477    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-node-log\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884496    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884526 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884533    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-run\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884512    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-sys\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcnv\" (UniqueName: \"kubernetes.io/projected/224cd297-364e-466b-a43e-3ac6a2de7d29-kube-api-access-bhcnv\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-bin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-multus\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-etc-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884593    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-ovn\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884650    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-run\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884682    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq9pz\" (UniqueName: \"kubernetes.io/projected/20572ffb-cc67-4066-b50c-22489eeed6e7-kube-api-access-fq9pz\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884684    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-bin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884708    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-ovn\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884712    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-etc-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-systemd\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884685    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-multus\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884765    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-tuned\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884773    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-systemd\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.884928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-socket-dir-parent\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884871    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-multus-daemon-config\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-hostroot\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884909    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-socket-dir-parent\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4b4m\" (UniqueName: \"kubernetes.io/projected/b202127e-afab-4cac-bee8-e93533bc25d5-kube-api-access-n4b4m\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884943    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-hostroot\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwnlq\" (UniqueName: \"kubernetes.io/projected/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-kube-api-access-zwnlq\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.884990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-slash\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885024    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-slash\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-netns\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-log-socket\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885097    2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885129    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-netns\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885131    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-multus-certs\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885159    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-log-socket\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-etc-kubernetes\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885191    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-etc-kubernetes\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.885663 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-tmp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885224    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-iptables-alerter-script\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4075d5d-2d2d-4e0e-b913-12715719e711-serviceca\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885261    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-conf\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m4sr\" (UniqueName: \"kubernetes.io/projected/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-kube-api-access-8m4sr\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-cni-binary-copy\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-conf-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885169    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-multus-certs\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885385    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-cnibin\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-bin\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20572ffb-cc67-4066-b50c-22489eeed6e7-ovn-node-metrics-cert\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-var-lib-kubelet\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:36.886347 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-cnibin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885620    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-conf\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-kubelet-config\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885689    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4075d5d-2d2d-4e0e-b913-12715719e711-serviceca\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885793    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885833    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-iptables-alerter-script\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885894    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-kubelet-config\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kppzz\" (UniqueName: \"kubernetes.io/projected/b4075d5d-2d2d-4e0e-b913-12715719e711-kube-api-access-kppzz\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885935    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-env-overrides\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.885991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-kubernetes\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-host-slash\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-dbus\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-hosts-file\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886156    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-var-lib-kubelet\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.887198 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-tmp-dir\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886187    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh6dp\" (UniqueName: \"kubernetes.io/projected/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-kube-api-access-vh6dp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-systemd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886239    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-cnibin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.886253    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886265    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-socket-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-registration-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.886339    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret podName:6e3b2c8e-e1e8-499f-9c8b-164e47a57f54 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:37.386319823 +0000 UTC m=+3.289905424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret") pod "global-pull-secret-syncer-kml2f" (UID: "6e3b2c8e-e1e8-499f-9c8b-164e47a57f54") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886355    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-registration-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-sys-fs\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-modprobe-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.886409    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886192    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-hosts-file\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.887980 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.886445    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs podName:224cd297-364e-466b-a43e-3ac6a2de7d29 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:37.386432495 +0000 UTC m=+3.290018084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs") pod "network-metrics-daemon-29t9c" (UID: "224cd297-364e-466b-a43e-3ac6a2de7d29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-kubelet\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886468    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-tmp-dir\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-var-lib-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886524    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-systemd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886531    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-var-lib-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-config\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-device-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886575    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-sys-fs\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886609    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-cnibin\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886620    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-socket-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-konnectivity-ca\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886651    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-system-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-lib-modules\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886684    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-host-slash\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-k8s-cni-cncf-io\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.888684 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886720    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-conf-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886732    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-device-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-netd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887028    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-env-overrides\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.886681    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-modprobe-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887094    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-bin\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887098    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-script-lib\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887141    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-lib-modules\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gmll\" (UniqueName: \"kubernetes.io/projected/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kube-api-access-2gmll\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysconfig\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-agent-certs\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887230    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-netd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m82st\" (UniqueName: \"kubernetes.io/projected/20a81c77-50d1-4e5b-a59d-43d455fb842d-kube-api-access-m82st\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887200    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-kubernetes\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.889310 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887315    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-dbus\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887342    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887378    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4075d5d-2d2d-4e0e-b913-12715719e711-host\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887420    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-system-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-systemd-units\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887517    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-systemd-units\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887546    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4075d5d-2d2d-4e0e-b913-12715719e711-host\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887589    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysconfig\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887622    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-k8s-cni-cncf-io\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.887648    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-kubelet\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.888554    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-config\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.888643    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-konnectivity-ca\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.888729    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.888989    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-script-lib\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.889147    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.889202    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-tuned\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.889258    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-tmp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.889852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.889312    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20572ffb-cc67-4066-b50c-22489eeed6e7-ovn-node-metrics-cert\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.890343 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.889379    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-cni-binary-copy\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.890617 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.890601    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-agent-certs\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:36.942977 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.942947    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kppzz\" (UniqueName: \"kubernetes.io/projected/b4075d5d-2d2d-4e0e-b913-12715719e711-kube-api-access-kppzz\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:36.943098 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.943068    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m4sr\" (UniqueName: \"kubernetes.io/projected/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-kube-api-access-8m4sr\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:36.943150 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.943106    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwnlq\" (UniqueName: \"kubernetes.io/projected/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-kube-api-access-zwnlq\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:36.944746 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.944201    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4b4m\" (UniqueName: \"kubernetes.io/projected/b202127e-afab-4cac-bee8-e93533bc25d5-kube-api-access-n4b4m\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:36.944746 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.944294    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhcnv\" (UniqueName: \"kubernetes.io/projected/224cd297-364e-466b-a43e-3ac6a2de7d29-kube-api-access-bhcnv\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:36.945331 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.945309    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq9pz\" (UniqueName: \"kubernetes.io/projected/20572ffb-cc67-4066-b50c-22489eeed6e7-kube-api-access-fq9pz\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:36.945919 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.945903    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:36.945998 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.945921    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:36.945998 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.945934    2577 projected.go:196] Error preparing data for projected volume kube-api-access-74qsz for pod openshift-network-diagnostics/network-check-target-f8whs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:36.945998 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:36.945996    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz podName:89987b8e-5b3f-4d20-b82e-643d32d7c004 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:37.445978475 +0000 UTC m=+3.349564079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-74qsz" (UniqueName: "kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz") pod "network-check-target-f8whs" (UID: "89987b8e-5b3f-4d20-b82e-643d32d7c004") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:36.950555 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.950537    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gmll\" (UniqueName: \"kubernetes.io/projected/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kube-api-access-2gmll\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:36.950790 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.950768    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh6dp\" (UniqueName: \"kubernetes.io/projected/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-kube-api-access-vh6dp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:36.952191 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.952173    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m82st\" (UniqueName: \"kubernetes.io/projected/20a81c77-50d1-4e5b-a59d-43d455fb842d-kube-api-access-m82st\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:36.971660 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:36.971640    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:37.060085 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.060054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:37.062239 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.062222    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:37.065573 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.065551    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9jtw5"
Oct 30 18:45:37.070515 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.070492    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6b70f0_1c02_4f23_bb51_b5cac08fa4dd.slice/crio-c1f497db70bfc0bd39ee6ff3870b5fdc586c37138c25460d59bc5cc5ae1cdd0c WatchSource:0}: Error finding container c1f497db70bfc0bd39ee6ff3870b5fdc586c37138c25460d59bc5cc5ae1cdd0c: Status 404 returned error can't find the container with id c1f497db70bfc0bd39ee6ff3870b5fdc586c37138c25460d59bc5cc5ae1cdd0c
Oct 30 18:45:37.071580 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.071558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:45:37.074210 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.074191    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20572ffb_cc67_4066_b50c_22489eeed6e7.slice/crio-c12213cefbad597648aca4a0b3a428db72476e55a88d8df3bcc206128326cbc4 WatchSource:0}: Error finding container c12213cefbad597648aca4a0b3a428db72476e55a88d8df3bcc206128326cbc4: Status 404 returned error can't find the container with id c12213cefbad597648aca4a0b3a428db72476e55a88d8df3bcc206128326cbc4
Oct 30 18:45:37.076191 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.076149    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb202127e_afab_4cac_bee8_e93533bc25d5.slice/crio-eda69974865db84652f4b9fe0fb6d591f9453fb23888e4d332d3a7b7bce95944 WatchSource:0}: Error finding container eda69974865db84652f4b9fe0fb6d591f9453fb23888e4d332d3a7b7bce95944: Status 404 returned error can't find the container with id eda69974865db84652f4b9fe0fb6d591f9453fb23888e4d332d3a7b7bce95944
Oct 30 18:45:37.081863 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.081845    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:45:37.082115 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.082080    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a93e14_0292_45a9_8e4b_b5a130ed3d55.slice/crio-8cb7324d074e007fe1f25f559bfd510fdf3c214b75716ffdc895dc48bac439c5 WatchSource:0}: Error finding container 8cb7324d074e007fe1f25f559bfd510fdf3c214b75716ffdc895dc48bac439c5: Status 404 returned error can't find the container with id 8cb7324d074e007fe1f25f559bfd510fdf3c214b75716ffdc895dc48bac439c5
Oct 30 18:45:37.083155 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.083138    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:45:37.086855 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.086834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:45:37.091700 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.091678    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a81c77_50d1_4e5b_a59d_43d455fb842d.slice/crio-8f30f6336b2dbfa9f8ab1c3ebbf735e06d291b8723261dd9aea1515c83f7c308 WatchSource:0}: Error finding container 8f30f6336b2dbfa9f8ab1c3ebbf735e06d291b8723261dd9aea1515c83f7c308: Status 404 returned error can't find the container with id 8f30f6336b2dbfa9f8ab1c3ebbf735e06d291b8723261dd9aea1515c83f7c308
Oct 30 18:45:37.092255 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.092240    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:45:37.093212 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.093181    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d3f34a6_a58f_44d7_9cf1_8e9a5f8c300b.slice/crio-44077ea33de44733087b5f4f941b3e27212c4c5c3b97af6a70b299fb6c395ee5 WatchSource:0}: Error finding container 44077ea33de44733087b5f4f941b3e27212c4c5c3b97af6a70b299fb6c395ee5: Status 404 returned error can't find the container with id 44077ea33de44733087b5f4f941b3e27212c4c5c3b97af6a70b299fb6c395ee5
Oct 30 18:45:37.096616 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.096588    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dce46b5_9bcf_4b26_8126_cac8319d7a9a.slice/crio-0267d1961df22fac98e440ea5167ed61574d1fb617f5817dd360955ee2202b74 WatchSource:0}: Error finding container 0267d1961df22fac98e440ea5167ed61574d1fb617f5817dd360955ee2202b74: Status 404 returned error can't find the container with id 0267d1961df22fac98e440ea5167ed61574d1fb617f5817dd360955ee2202b74
Oct 30 18:45:37.096616 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.096612    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:45:37.102379 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.102355    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4075d5d_2d2d_4e0e_b913_12715719e711.slice/crio-9398e679b4eaef3f2b9f8d16b4395486a09a9ecc85e2e2c21edb803c0cde8ad5 WatchSource:0}: Error finding container 9398e679b4eaef3f2b9f8d16b4395486a09a9ecc85e2e2c21edb803c0cde8ad5: Status 404 returned error can't find the container with id 9398e679b4eaef3f2b9f8d16b4395486a09a9ecc85e2e2c21edb803c0cde8ad5
Oct 30 18:45:37.106125 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:45:37.106094    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbeaac75_73d1_4d21_ad63_7215bb75e8e6.slice/crio-a92d5376dba793ee1a873662d6b3cfe8968b2fd4ad2177a12f54e3a4587332a5 WatchSource:0}: Error finding container a92d5376dba793ee1a873662d6b3cfe8968b2fd4ad2177a12f54e3a4587332a5: Status 404 returned error can't find the container with id a92d5376dba793ee1a873662d6b3cfe8968b2fd4ad2177a12f54e3a4587332a5
Oct 30 18:45:37.391114 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.391039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:37.391114 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.391085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:37.391323 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:37.391230    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:37.391323 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:37.391292    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret podName:6e3b2c8e-e1e8-499f-9c8b-164e47a57f54 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:38.39127453 +0000 UTC m=+4.294860122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret") pod "global-pull-secret-syncer-kml2f" (UID: "6e3b2c8e-e1e8-499f-9c8b-164e47a57f54") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:37.391433 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:37.391362    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:37.391433 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:37.391398    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs podName:224cd297-364e-466b-a43e-3ac6a2de7d29 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:38.391386489 +0000 UTC m=+4.294972078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs") pod "network-metrics-daemon-29t9c" (UID: "224cd297-364e-466b-a43e-3ac6a2de7d29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:37.492093 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.492058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:37.492271 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:37.492206    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:37.492271 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:37.492227    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:37.492271 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:37.492239    2577 projected.go:196] Error preparing data for projected volume kube-api-access-74qsz for pod openshift-network-diagnostics/network-check-target-f8whs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:37.492413 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:37.492293    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz podName:89987b8e-5b3f-4d20-b82e-643d32d7c004 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:38.492275798 +0000 UTC m=+4.395861394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-74qsz" (UniqueName: "kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz") pod "network-check-target-f8whs" (UID: "89987b8e-5b3f-4d20-b82e-643d32d7c004") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:37.658058 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.657977    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6rdvd" event={"ID":"bbeaac75-73d1-4d21-ad63-7215bb75e8e6","Type":"ContainerStarted","Data":"a92d5376dba793ee1a873662d6b3cfe8968b2fd4ad2177a12f54e3a4587332a5"}
Oct 30 18:45:37.659456 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.659428    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ns79r" event={"ID":"20a81c77-50d1-4e5b-a59d-43d455fb842d","Type":"ContainerStarted","Data":"8f30f6336b2dbfa9f8ab1c3ebbf735e06d291b8723261dd9aea1515c83f7c308"}
Oct 30 18:45:37.660976 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.660914    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd" event={"ID":"96a93e14-0292-45a9-8e4b-b5a130ed3d55","Type":"ContainerStarted","Data":"8cb7324d074e007fe1f25f559bfd510fdf3c214b75716ffdc895dc48bac439c5"}
Oct 30 18:45:37.664227 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.664192    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9jtw5" event={"ID":"b202127e-afab-4cac-bee8-e93533bc25d5","Type":"ContainerStarted","Data":"eda69974865db84652f4b9fe0fb6d591f9453fb23888e4d332d3a7b7bce95944"}
Oct 30 18:45:37.666450 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.666357    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"c12213cefbad597648aca4a0b3a428db72476e55a88d8df3bcc206128326cbc4"}
Oct 30 18:45:37.667823 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.667757    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xhv7g" event={"ID":"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd","Type":"ContainerStarted","Data":"c1f497db70bfc0bd39ee6ff3870b5fdc586c37138c25460d59bc5cc5ae1cdd0c"}
Oct 30 18:45:37.669210 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.669184    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-phfz4" event={"ID":"b4075d5d-2d2d-4e0e-b913-12715719e711","Type":"ContainerStarted","Data":"9398e679b4eaef3f2b9f8d16b4395486a09a9ecc85e2e2c21edb803c0cde8ad5"}
Oct 30 18:45:37.670380 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.670353    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sg2mz" event={"ID":"0dce46b5-9bcf-4b26-8126-cac8319d7a9a","Type":"ContainerStarted","Data":"0267d1961df22fac98e440ea5167ed61574d1fb617f5817dd360955ee2202b74"}
Oct 30 18:45:37.671485 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.671462    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t75nr" event={"ID":"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b","Type":"ContainerStarted","Data":"44077ea33de44733087b5f4f941b3e27212c4c5c3b97af6a70b299fb6c395ee5"}
Oct 30 18:45:37.843340 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.843261    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:40:36 +0000 UTC" deadline="2028-07-11 01:55:14.950500776 +0000 UTC"
Oct 30 18:45:37.843340 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:37.843336    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14863h9m37.107169238s"
Oct 30 18:45:38.398751 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:38.398713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:38.398956 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:38.398771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:38.398956 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.398939    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:38.399063 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.399009    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret podName:6e3b2c8e-e1e8-499f-9c8b-164e47a57f54 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:40.398989626 +0000 UTC m=+6.302575229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret") pod "global-pull-secret-syncer-kml2f" (UID: "6e3b2c8e-e1e8-499f-9c8b-164e47a57f54") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:38.399125 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.399081    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:38.399125 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.399124    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs podName:224cd297-364e-466b-a43e-3ac6a2de7d29 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:40.399110707 +0000 UTC m=+6.302696297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs") pod "network-metrics-daemon-29t9c" (UID: "224cd297-364e-466b-a43e-3ac6a2de7d29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:38.499883 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:38.499833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:38.500071 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.499961    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:38.500071 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.499990    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:38.500071 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.500005    2577 projected.go:196] Error preparing data for projected volume kube-api-access-74qsz for pod openshift-network-diagnostics/network-check-target-f8whs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:38.500071 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.500070    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz podName:89987b8e-5b3f-4d20-b82e-643d32d7c004 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:40.500050704 +0000 UTC m=+6.403636284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-74qsz" (UniqueName: "kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz") pod "network-check-target-f8whs" (UID: "89987b8e-5b3f-4d20-b82e-643d32d7c004") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:38.650950 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:38.650871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:38.651099 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.651020    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:38.651160 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:38.651144    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:38.651257 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.651237    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:38.651318 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:38.651311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:38.651448 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:38.651416    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:38.844501 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:38.844455    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:40:36 +0000 UTC" deadline="2028-06-27 03:58:18.937561271 +0000 UTC"
Oct 30 18:45:38.844501 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:38.844497    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14529h12m40.093068794s"
Oct 30 18:45:40.418391 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.417743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:40.418391 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.418011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:40.418391 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.418160    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:40.418391 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.418219    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret podName:6e3b2c8e-e1e8-499f-9c8b-164e47a57f54 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:44.418200848 +0000 UTC m=+10.321786440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret") pod "global-pull-secret-syncer-kml2f" (UID: "6e3b2c8e-e1e8-499f-9c8b-164e47a57f54") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:40.418391 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.418284    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:40.418391 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.418319    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs podName:224cd297-364e-466b-a43e-3ac6a2de7d29 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:44.418306898 +0000 UTC m=+10.321892487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs") pod "network-metrics-daemon-29t9c" (UID: "224cd297-364e-466b-a43e-3ac6a2de7d29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:40.519300 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.519035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:40.519300 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.519176    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:40.519300 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.519193    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:40.519300 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.519202    2577 projected.go:196] Error preparing data for projected volume kube-api-access-74qsz for pod openshift-network-diagnostics/network-check-target-f8whs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:40.519300 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.519246    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz podName:89987b8e-5b3f-4d20-b82e-643d32d7c004 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:44.519234168 +0000 UTC m=+10.422819756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-74qsz" (UniqueName: "kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz") pod "network-check-target-f8whs" (UID: "89987b8e-5b3f-4d20-b82e-643d32d7c004") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:40.650420 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.650345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:40.650587 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.650485    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:40.650587 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.650355    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:40.650707 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.650640    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:40.650707 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.650355    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:40.650828 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:40.650717    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:40.692790 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.692746    2577 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="a34ce64c298b4597d0f9c7ed8f78e0a66807859d8b4d6f66aae657a560a84eeb" exitCode=0
Oct 30 18:45:40.692972 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.692879    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ns79r" event={"ID":"20a81c77-50d1-4e5b-a59d-43d455fb842d","Type":"ContainerDied","Data":"a34ce64c298b4597d0f9c7ed8f78e0a66807859d8b4d6f66aae657a560a84eeb"}
Oct 30 18:45:40.699604 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.698979    2577 generic.go:356] "Generic (PLEG): container finished" podID="58894da434aca988e2149b0960c93e80" containerID="244a61cf726bb4b7443b3894878eab0faacf28a0c1f0f3359945ce70f5bed6b6" exitCode=0
Oct 30 18:45:40.699604 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.699064    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"58894da434aca988e2149b0960c93e80","Type":"ContainerDied","Data":"244a61cf726bb4b7443b3894878eab0faacf28a0c1f0f3359945ce70f5bed6b6"}
Oct 30 18:45:40.712257 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:40.711737    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"d0bf5677d5ce41c447fb39800129b147","Type":"ContainerStarted","Data":"bc23fcbd9fd02337e3d3aee7f797f4acaae7754c0b3e06cc57621a0c43a1b162"}
Oct 30 18:45:41.122172 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:41.122074    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal" podStartSLOduration=5.12205436 podStartE2EDuration="5.12205436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:41.014176877 +0000 UTC m=+6.917762475" watchObservedRunningTime="2026-10-30 18:45:41.12205436 +0000 UTC m=+7.025639958"
Oct 30 18:45:41.723647 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:41.722874    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"58894da434aca988e2149b0960c93e80","Type":"ContainerStarted","Data":"f6ce8d1861ca09ab9ad591a308d68fc925ae2872caaee6b4131907c4f1e25c2a"}
Oct 30 18:45:41.729079 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:41.729001    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xhv7g" event={"ID":"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd","Type":"ContainerStarted","Data":"a3177657b2b6861708d6d7239ec10af95e880656868083af276bc587ed555b0b"}
Oct 30 18:45:41.940876 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:41.940793    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal" podStartSLOduration=5.940773043 podStartE2EDuration="5.940773043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:41.832441319 +0000 UTC m=+7.736026920" watchObservedRunningTime="2026-10-30 18:45:41.940773043 +0000 UTC m=+7.844358642"
Oct 30 18:45:42.651713 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:42.651287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:42.651713 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:42.651428    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:42.651713 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:42.651499    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:42.651713 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:42.651556    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:42.651713 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:42.651607    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:42.651713 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:42.651677    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:43.737946 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:43.737863    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd" event={"ID":"96a93e14-0292-45a9-8e4b-b5a130ed3d55","Type":"ContainerStarted","Data":"de7982650e3c463bf84075bd2e96687d09cec6e855c285d6b9f3a0ef07857f86"}
Oct 30 18:45:43.741612 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:43.741584    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-phfz4" event={"ID":"b4075d5d-2d2d-4e0e-b913-12715719e711","Type":"ContainerStarted","Data":"4e11200560bcf59a54c6407429baa59ac6953e41be6a1518f3abe90864881076"}
Oct 30 18:45:43.849734 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:43.849668    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xhv7g" podStartSLOduration=5.634658386 podStartE2EDuration="9.84964928s" totalImagesPullingTime="4.214990894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:41.941418298 +0000 UTC m=+7.845003897" watchObservedRunningTime="2026-10-30 18:45:43.84964928 +0000 UTC m=+9.753234879"
Oct 30 18:45:44.327944 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.327913    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:44.329211 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.329017    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:44.438386 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.437438    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-phfz4" podStartSLOduration=3.717281526 podStartE2EDuration="9.437418316s" totalImagesPullingTime="5.72013679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:43.850151413 +0000 UTC m=+9.753737013" watchObservedRunningTime="2026-10-30 18:45:44.437418316 +0000 UTC m=+10.341003913"
Oct 30 18:45:44.461095 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.461061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:44.461262 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.461111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:44.461558 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.461541    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:44.461640 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.461601    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs podName:224cd297-364e-466b-a43e-3ac6a2de7d29 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:52.461584713 +0000 UTC m=+18.365170304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs") pod "network-metrics-daemon-29t9c" (UID: "224cd297-364e-466b-a43e-3ac6a2de7d29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:44.461702 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.461664    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:44.461702 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.461696    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret podName:6e3b2c8e-e1e8-499f-9c8b-164e47a57f54 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:52.461685062 +0000 UTC m=+18.365270652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret") pod "global-pull-secret-syncer-kml2f" (UID: "6e3b2c8e-e1e8-499f-9c8b-164e47a57f54") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:44.561936 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.561900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:44.562142 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.562101    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:44.562142 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.562119    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:44.562142 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.562131    2577 projected.go:196] Error preparing data for projected volume kube-api-access-74qsz for pod openshift-network-diagnostics/network-check-target-f8whs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:44.562351 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.562182    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz podName:89987b8e-5b3f-4d20-b82e-643d32d7c004 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:52.562168776 +0000 UTC m=+18.465754363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-74qsz" (UniqueName: "kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz") pod "network-check-target-f8whs" (UID: "89987b8e-5b3f-4d20-b82e-643d32d7c004") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:44.649721 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.649657    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:44.649902 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.649768    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:44.649902 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.649846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:44.649991 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.649927    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:44.649991 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:44.649978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:44.650088 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:44.650042    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:45.520335 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:45.520307    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:45.521979 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:45.521960    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:45:45.529295 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:45.529256    2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:45:45.642725 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:45.642585    2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:45:45.529270593Z","UUID":"df78e0cf-1943-4786-b905-80853fc5106a","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:45:45.644304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:45.644280    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:45:45.644408 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:45.644314    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:45:45.754211 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:45.754171    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd" event={"ID":"96a93e14-0292-45a9-8e4b-b5a130ed3d55","Type":"ContainerStarted","Data":"87e9e29ce2c8e4293093768e0ec0256d339d468196863938d31e11daf31defbc"}
Oct 30 18:45:45.757497 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:45.757338    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9jtw5" event={"ID":"b202127e-afab-4cac-bee8-e93533bc25d5","Type":"ContainerStarted","Data":"93799929d9f910de6b2f32059c4083c46fbea27ce2928d1b881b980714396333"}
Oct 30 18:45:46.651398 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:46.651368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:46.651852 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:46.651487    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:46.651852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:46.651788    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:46.651982 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:46.651902    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:46.651982 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:46.651949    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:46.652083 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:46.652053    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:46.765097 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:46.765060    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6rdvd" event={"ID":"bbeaac75-73d1-4d21-ad63-7215bb75e8e6","Type":"ContainerStarted","Data":"a58bdd74087b194bf9cf23f6b0fbabe962338066db7d52e46c76584bdb1a6eaf"}
Oct 30 18:45:46.767393 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:46.767363    2577 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="9c45be752cc8b07f1f1a9434ea858d58e4eedd7cd5842f8e3c275998fd80de54" exitCode=0
Oct 30 18:45:46.768257 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:46.768227    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ns79r" event={"ID":"20a81c77-50d1-4e5b-a59d-43d455fb842d","Type":"ContainerDied","Data":"9c45be752cc8b07f1f1a9434ea858d58e4eedd7cd5842f8e3c275998fd80de54"}
Oct 30 18:45:46.874359 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:46.874137    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-9jtw5" podStartSLOduration=4.814250298 podStartE2EDuration="12.874117334s" totalImagesPullingTime="8.059867036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:45.869670425 +0000 UTC m=+11.773256023" watchObservedRunningTime="2026-10-30 18:45:46.874117334 +0000 UTC m=+12.777702933"
Oct 30 18:45:46.988871 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:46.988764    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-6rdvd" podStartSLOduration=3.493322016 podStartE2EDuration="11.988749095s" totalImagesPullingTime="8.495427079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:46.873874983 +0000 UTC m=+12.777460580" watchObservedRunningTime="2026-10-30 18:45:46.988749095 +0000 UTC m=+12.892334697"
Oct 30 18:45:47.776148 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:47.776113    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd" event={"ID":"96a93e14-0292-45a9-8e4b-b5a130ed3d55","Type":"ContainerStarted","Data":"ae5c95dce3cd262edd9738eb7447ea5c055bcc2846e6c6f81c1df573b32a0b0d"}
Oct 30 18:45:47.777921 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:47.777891    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sg2mz" event={"ID":"0dce46b5-9bcf-4b26-8126-cac8319d7a9a","Type":"ContainerStarted","Data":"3a38f916feafc39432fb62bc71e1462f2b39e8f6d6893f56d41d2b251f980525"}
Oct 30 18:45:48.000388 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:48.000329    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd" podStartSLOduration=3.988200578 podStartE2EDuration="14.000309551s" totalImagesPullingTime="10.012108973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:47.892868015 +0000 UTC m=+13.796453614" watchObservedRunningTime="2026-10-30 18:45:48.000309551 +0000 UTC m=+13.903895154"
Oct 30 18:45:48.000574 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:48.000528    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sg2mz" podStartSLOduration=5.7961203359999995 podStartE2EDuration="14.000519967s" totalImagesPullingTime="8.204399631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:48.000087185 +0000 UTC m=+13.903672809" watchObservedRunningTime="2026-10-30 18:45:48.000519967 +0000 UTC m=+13.904105578"
Oct 30 18:45:48.649338 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:48.649264    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:48.649338 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:48.649297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:48.649338 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:48.649264    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:48.649598 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:48.649408    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:48.649598 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:48.649432    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:48.649598 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:48.649505    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:48.781566 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:48.781521    2577 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="9545194544c4c1e44e4b71177f094d0ab364432deb1a7ed442c59e9331f9e2df" exitCode=0
Oct 30 18:45:48.782206 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:48.781611    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ns79r" event={"ID":"20a81c77-50d1-4e5b-a59d-43d455fb842d","Type":"ContainerDied","Data":"9545194544c4c1e44e4b71177f094d0ab364432deb1a7ed442c59e9331f9e2df"}
Oct 30 18:45:48.783341 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:48.783317    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t75nr" event={"ID":"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b","Type":"ContainerStarted","Data":"f349b5b652a1dd81abcdb74620afa362510e8f2a72c51831989406ccf21e71cd"}
Oct 30 18:45:49.055789 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:49.055698    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t75nr" podStartSLOduration=4.39236661 podStartE2EDuration="15.055678835s" totalImagesPullingTime="10.663312225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:49.055407225 +0000 UTC m=+14.958992836" watchObservedRunningTime="2026-10-30 18:45:49.055678835 +0000 UTC m=+14.959264433"
Oct 30 18:45:50.648828 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:50.648778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:50.649361 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:50.648778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:50.649361 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:50.648916    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:50.649361 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:50.648795    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:50.649361 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:50.648998    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:50.649361 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:50.649050    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:50.791414 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:50.791216    2577 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="ef188b1fdc7a347c84532ced574f6f88bffff31ebc3e28caeddbd21734d5f477" exitCode=0
Oct 30 18:45:50.791583 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:50.791312    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ns79r" event={"ID":"20a81c77-50d1-4e5b-a59d-43d455fb842d","Type":"ContainerDied","Data":"ef188b1fdc7a347c84532ced574f6f88bffff31ebc3e28caeddbd21734d5f477"}
Oct 30 18:45:52.523021 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.522935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:52.523021 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.522990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:52.523493 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.523121    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:52.523493 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.523135    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:52.523493 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.523190    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret podName:6e3b2c8e-e1e8-499f-9c8b-164e47a57f54 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:08.523173013 +0000 UTC m=+34.426758589 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret") pod "global-pull-secret-syncer-kml2f" (UID: "6e3b2c8e-e1e8-499f-9c8b-164e47a57f54") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:52.523493 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.523215    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs podName:224cd297-364e-466b-a43e-3ac6a2de7d29 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:08.523204727 +0000 UTC m=+34.426790309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs") pod "network-metrics-daemon-29t9c" (UID: "224cd297-364e-466b-a43e-3ac6a2de7d29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:52.623319 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.623288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:52.623477 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.623426    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:52.623477 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.623449    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:52.623477 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.623460    2577 projected.go:196] Error preparing data for projected volume kube-api-access-74qsz for pod openshift-network-diagnostics/network-check-target-f8whs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:52.623581 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.623519    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz podName:89987b8e-5b3f-4d20-b82e-643d32d7c004 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:08.623501484 +0000 UTC m=+34.527087062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-74qsz" (UniqueName: "kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz") pod "network-check-target-f8whs" (UID: "89987b8e-5b3f-4d20-b82e-643d32d7c004") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:52.649719 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.649676    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:52.649719 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.649699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:52.649925 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.649780    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:52.649925 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.649900    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:52.649999 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.649984    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:52.650081 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:52.650058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:52.798945 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.798913    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"68506dad0328155fdc4f37f85fc2e779803b7479ee78b87bbcad69674346e69c"}
Oct 30 18:45:52.799053 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.798985    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"bec54a89122f0f4658336aa438ad07a2b5075d95003e00381ed8880628b74497"}
Oct 30 18:45:52.799053 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.799016    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"bb3ffae52e001d839fd5cd79684b018bc816b47a136b77745f155af2c2c83b3c"}
Oct 30 18:45:52.799053 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.799047    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"291f63d7911f8979f69ff1e317ae0bcc260882724021c1f0151d8e2f5ef070c2"}
Oct 30 18:45:52.799145 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.799072    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"d2d6927ac5fdf9d3443f5879357bb549532971b5629f862642d21d4ed15ee214"}
Oct 30 18:45:52.799145 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:52.799097    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"230a9100e46ee24accb9a4600f868540700497692c319ba4926b07ab6358ec60"}
Oct 30 18:45:53.805457 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:53.805417    2577 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="365049c1f711bff37e6886c83f4b9d1acf0767f893e068687b8c81a7c312e136" exitCode=0
Oct 30 18:45:53.805830 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:53.805500    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ns79r" event={"ID":"20a81c77-50d1-4e5b-a59d-43d455fb842d","Type":"ContainerDied","Data":"365049c1f711bff37e6886c83f4b9d1acf0767f893e068687b8c81a7c312e136"}
Oct 30 18:45:54.650421 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:54.650335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:54.650421 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:54.650421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:54.650592 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:54.650466    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:54.650592 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:54.650479    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:54.650592 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:54.650541    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:54.650711 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:54.650636    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:54.809893 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:54.809862    2577 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="3ab96e0fac7e10ce6a271f9b974090048dc08bed049a0de10ca812600d27acb3" exitCode=0
Oct 30 18:45:54.810379 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:54.809951    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ns79r" event={"ID":"20a81c77-50d1-4e5b-a59d-43d455fb842d","Type":"ContainerDied","Data":"3ab96e0fac7e10ce6a271f9b974090048dc08bed049a0de10ca812600d27acb3"}
Oct 30 18:45:54.813143 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:54.813121    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"ad4f40aa161e4b8cbc4f4e7e92d36f40656f78079a56ed4074b5c4ec3779818e"}
Oct 30 18:45:55.818478 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:55.818445    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ns79r" event={"ID":"20a81c77-50d1-4e5b-a59d-43d455fb842d","Type":"ContainerStarted","Data":"e5dd48775428bc067926f87b4557f7a56c9eb679b6444e3177cc527c6f803596"}
Oct 30 18:45:56.648852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:56.648803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:56.648852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:56.648833    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:56.649146 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:56.648908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:56.649146 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:56.649009    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:56.649146 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:56.649114    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:56.649283 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:56.649190    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:56.823806 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:56.823719    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" event={"ID":"20572ffb-cc67-4066-b50c-22489eeed6e7","Type":"ContainerStarted","Data":"8d574ec892ff56e0826cc70cb35e2f9b9fe7b71dc40035abca2738e87063e931"}
Oct 30 18:45:57.083670 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:57.083413    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9" podStartSLOduration=8.171766057 podStartE2EDuration="23.08339789s" totalImagesPullingTime="14.911631833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:57.03722542 +0000 UTC m=+22.940811016" watchObservedRunningTime="2026-10-30 18:45:57.08339789 +0000 UTC m=+22.986983480"
Oct 30 18:45:57.084047 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:57.084019    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ns79r" podStartSLOduration=10.271127051 podStartE2EDuration="23.084013157s" totalImagesPullingTime="12.812886106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:55.938805923 +0000 UTC m=+21.842391521" watchObservedRunningTime="2026-10-30 18:45:57.084013157 +0000 UTC m=+22.987598753"
Oct 30 18:45:57.825648 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:57.825610    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:57.826179 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:57.826139    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:57.826247 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:57.826227    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:57.842251 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:57.842222    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:57.843176 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:57.843158    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:45:58.648798 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:58.648762    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:58.648798 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:58.648778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:58.649019 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:58.648805    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:58.649019 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:58.648897    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:58.649104 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:58.649020    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:58.649104 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:58.649087    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:45:59.240641 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:59.240617    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29t9c"]
Oct 30 18:45:59.241030 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:59.240714    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:45:59.241030 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:59.240830    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:45:59.241124 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:59.241102    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kml2f"]
Oct 30 18:45:59.241186 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:59.241174    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:45:59.241278 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:59.241257    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:45:59.243679 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:59.243661    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f8whs"]
Oct 30 18:45:59.243830 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:45:59.243724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:45:59.243830 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:45:59.243783    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:00.649127 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:00.649090    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:00.649127 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:00.649121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:00.649679 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:00.649091    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:00.649679 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:00.649193    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:00.649679 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:00.649270    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:00.649679 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:00.649337    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:02.648981 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:02.648945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:02.648981 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:02.648976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:02.649428 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:02.648988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:02.649428 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:02.649057    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:02.649428 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:02.649139    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:02.649428 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:02.649238    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:03.691857 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:03.691745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xhv7g_cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:04.650133 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:04.650107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:04.650294 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:04.650195    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:04.650346 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:04.650293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:04.650421 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:04.650407    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:04.650475 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:04.650425    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:04.650531 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:04.650510    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:05.076699 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:05.076669    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6rdvd_bbeaac75-73d1-4d21-ad63-7215bb75e8e6/dns-node-resolver/0.log"
Oct 30 18:46:06.477847 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:06.477805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-phfz4_b4075d5d-2d2d-4e0e-b913-12715719e711/node-ca/0.log"
Oct 30 18:46:06.649462 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:06.649427    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:06.649622 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:06.649478    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:06.649622 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:06.649496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:06.649622 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:06.649566    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:06.649735 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:06.649645    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:06.649735 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:06.649714    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:08.533422 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:08.533394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:08.533837 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:08.533437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:08.533837 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.533545    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:08.533837 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.533566    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:08.533837 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.533607    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs podName:224cd297-364e-466b-a43e-3ac6a2de7d29 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:40.533591995 +0000 UTC m=+66.437177596 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs") pod "network-metrics-daemon-29t9c" (UID: "224cd297-364e-466b-a43e-3ac6a2de7d29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:08.533837 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.533621    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret podName:6e3b2c8e-e1e8-499f-9c8b-164e47a57f54 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:40.533615026 +0000 UTC m=+66.437200601 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret") pod "global-pull-secret-syncer-kml2f" (UID: "6e3b2c8e-e1e8-499f-9c8b-164e47a57f54") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:08.634502 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:08.634469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:08.634659 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.634623    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:46:08.634659 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.634641    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:46:08.634659 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.634652    2577 projected.go:196] Error preparing data for projected volume kube-api-access-74qsz for pod openshift-network-diagnostics/network-check-target-f8whs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:46:08.634751 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.634698    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz podName:89987b8e-5b3f-4d20-b82e-643d32d7c004 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:40.634685703 +0000 UTC m=+66.538271282 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-74qsz" (UniqueName: "kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz") pod "network-check-target-f8whs" (UID: "89987b8e-5b3f-4d20-b82e-643d32d7c004") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:46:08.649351 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:08.649319    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:08.649485 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:08.649435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:08.649537 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.649517    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:08.649578 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.649428    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:08.649578 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:08.649543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:08.649645 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:08.649622    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:10.649719 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:10.649686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:10.650179 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:10.649698    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:10.650179 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:10.649837    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:10.650179 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:10.649898    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:10.650179 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:10.649792    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:10.650179 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:10.649999    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:12.649232 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:12.649204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:12.649705 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:12.649204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:12.649705 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:12.649324    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:12.649705 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:12.649426    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:12.649705 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:12.649204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:12.649705 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:12.649498    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:14.650005 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:14.649974    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:14.650384 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:14.650060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:14.650384 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:14.650092    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:14.650384 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:14.650105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:14.650384 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:14.650175    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:14.650384 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:14.650268    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:16.649044 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:16.649009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:16.649044 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:16.649037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:16.649458 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:16.649010    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:16.649458 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:16.649139    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:16.649458 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:16.649254    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:16.649458 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:16.649330    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:18.648712 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:18.648676    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:18.648712 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:18.648715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:18.649288 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:18.648685    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:18.649288 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:18.648795    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:18.649288 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:18.648868    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:18.649288 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:18.648946    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:20.648661 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:20.648634    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:20.649163 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:20.648641    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:20.649163 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:20.648735    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:20.649163 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:20.648645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:20.649163 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:20.648840    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:20.649163 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:20.648938    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:22.649668 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:22.649634    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:22.649668 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:22.649656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:22.650152 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:22.649635    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:22.650152 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:22.649751    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f8whs" podUID="89987b8e-5b3f-4d20-b82e-643d32d7c004"
Oct 30 18:46:22.650152 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:22.649837    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29t9c" podUID="224cd297-364e-466b-a43e-3ac6a2de7d29"
Oct 30 18:46:22.650152 ip-10-0-143-248 kubenswrapper[2577]: E1030 18:46:22.649894    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kml2f" podUID="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54"
Oct 30 18:46:23.317255 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.317228    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeReady"
Oct 30 18:46:23.317434 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.317354    2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:46:23.425695 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.425664    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zhb5w"]
Oct 30 18:46:23.427755 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.427739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.440382 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.440364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a768b8cc-62a2-4588-82df-edd68a625830-data-volume\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.440471 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.440389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqn7d\" (UniqueName: \"kubernetes.io/projected/a768b8cc-62a2-4588-82df-edd68a625830-kube-api-access-nqn7d\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.440471 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.440418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a768b8cc-62a2-4588-82df-edd68a625830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.440535 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.440507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a768b8cc-62a2-4588-82df-edd68a625830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.440572 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.440539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a768b8cc-62a2-4588-82df-edd68a625830-crio-socket\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.471688 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.471659    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cn7zd"]
Oct 30 18:46:23.473733 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.473717    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.478316 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.478293    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:46:23.478956 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.478937    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:46:23.479036 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.479011    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:46:23.479186 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.479170    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k86kp\""
Oct 30 18:46:23.479759 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.479720    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zhb5w"]
Oct 30 18:46:23.487514 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.487490    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cn7zd"]
Oct 30 18:46:23.517939 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.517903    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tkr8v"]
Oct 30 18:46:23.520164 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.520141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:23.540859 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.540832    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a768b8cc-62a2-4588-82df-edd68a625830-data-volume\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.540859 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.540862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqn7d\" (UniqueName: \"kubernetes.io/projected/a768b8cc-62a2-4588-82df-edd68a625830-kube-api-access-nqn7d\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.541121 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.540888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjq6z\" (UniqueName: \"kubernetes.io/projected/13af2e85-a46f-496a-9151-5c6d649e6fe5-kube-api-access-jjq6z\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.541121 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.540928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a768b8cc-62a2-4588-82df-edd68a625830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.541121 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/13af2e85-a46f-496a-9151-5c6d649e6fe5-config-volume\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.541121 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13af2e85-a46f-496a-9151-5c6d649e6fe5-tmp-dir\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.541121 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a768b8cc-62a2-4588-82df-edd68a625830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.541353 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541171    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a768b8cc-62a2-4588-82df-edd68a625830-data-volume\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.541353 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13af2e85-a46f-496a-9151-5c6d649e6fe5-metrics-tls\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.541619 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a768b8cc-62a2-4588-82df-edd68a625830-crio-socket\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.541799 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46605ee7-f737-431c-8a64-0b430a550d34-cert\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:23.541969 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j58xm\" (UniqueName: \"kubernetes.io/projected/46605ee7-f737-431c-8a64-0b430a550d34-kube-api-access-j58xm\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:23.541969 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.541937    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a768b8cc-62a2-4588-82df-edd68a625830-crio-socket\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.542121 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.542010    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a768b8cc-62a2-4588-82df-edd68a625830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.546008 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.545987    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a768b8cc-62a2-4588-82df-edd68a625830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.578614 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.578542    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tkr8v"]
Oct 30 18:46:23.578750 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.578735    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:46:23.579067 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.579040    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:46:23.579067 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.579064    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjgl9\""
Oct 30 18:46:23.579067 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.579069    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:46:23.579311 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.579134    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-vvlg2\""
Oct 30 18:46:23.579311 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.579244    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:46:23.579311 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.579243    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:46:23.579311 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.579265    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:46:23.604002 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.603969    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqn7d\" (UniqueName: \"kubernetes.io/projected/a768b8cc-62a2-4588-82df-edd68a625830-kube-api-access-nqn7d\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.643168 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.643132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j58xm\" (UniqueName: \"kubernetes.io/projected/46605ee7-f737-431c-8a64-0b430a550d34-kube-api-access-j58xm\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:23.643350 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.643196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjq6z\" (UniqueName: \"kubernetes.io/projected/13af2e85-a46f-496a-9151-5c6d649e6fe5-kube-api-access-jjq6z\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.643350 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.643221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/13af2e85-a46f-496a-9151-5c6d649e6fe5-config-volume\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.643350 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.643237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13af2e85-a46f-496a-9151-5c6d649e6fe5-tmp-dir\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.643350 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.643258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13af2e85-a46f-496a-9151-5c6d649e6fe5-metrics-tls\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.643350 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.643283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46605ee7-f737-431c-8a64-0b430a550d34-cert\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:23.643614 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.643594    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13af2e85-a46f-496a-9151-5c6d649e6fe5-tmp-dir\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.643884 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.643856    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/13af2e85-a46f-496a-9151-5c6d649e6fe5-config-volume\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.645553 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.645528    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13af2e85-a46f-496a-9151-5c6d649e6fe5-metrics-tls\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.645656 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.645533    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46605ee7-f737-431c-8a64-0b430a550d34-cert\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:23.701743 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.701699    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j58xm\" (UniqueName: \"kubernetes.io/projected/46605ee7-f737-431c-8a64-0b430a550d34-kube-api-access-j58xm\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:23.704044 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.704021    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjq6z\" (UniqueName: \"kubernetes.io/projected/13af2e85-a46f-496a-9151-5c6d649e6fe5-kube-api-access-jjq6z\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.740930 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.740900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:23.785112 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.785084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:23.829842 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:23.829754    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:24.162530 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.162500    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zhb5w"]
Oct 30 18:46:24.170357 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.170338    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cn7zd"]
Oct 30 18:46:24.207282 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:24.207258    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13af2e85_a46f_496a_9151_5c6d649e6fe5.slice/crio-a224647afab66b3dd230cd4bb5465c8f63d454a6b509c792c0a7fa0dcde13cac WatchSource:0}: Error finding container a224647afab66b3dd230cd4bb5465c8f63d454a6b509c792c0a7fa0dcde13cac: Status 404 returned error can't find the container with id a224647afab66b3dd230cd4bb5465c8f63d454a6b509c792c0a7fa0dcde13cac
Oct 30 18:46:24.215361 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.215338    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tkr8v"]
Oct 30 18:46:24.221472 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:24.221449    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46605ee7_f737_431c_8a64_0b430a550d34.slice/crio-09b151b0942db019c719857b6ffef310a633086971628d385da08d8345221265 WatchSource:0}: Error finding container 09b151b0942db019c719857b6ffef310a633086971628d385da08d8345221265: Status 404 returned error can't find the container with id 09b151b0942db019c719857b6ffef310a633086971628d385da08d8345221265
Oct 30 18:46:24.652082 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.652059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:24.652252 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.652092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:24.652252 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.652193    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:24.704718 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.704685    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:46:24.705179 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.704848    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:46:24.705364 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.705348    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:46:24.705436 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.705379    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q9tbn\""
Oct 30 18:46:24.705436 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.705380    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:46:24.705436 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.705430    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgxcg\""
Oct 30 18:46:24.705535 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.705472    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-crxtc\""
Oct 30 18:46:24.883830 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.883775    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tkr8v" event={"ID":"46605ee7-f737-431c-8a64-0b430a550d34","Type":"ContainerStarted","Data":"09b151b0942db019c719857b6ffef310a633086971628d385da08d8345221265"}
Oct 30 18:46:24.884842 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.884800    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn7zd" event={"ID":"13af2e85-a46f-496a-9151-5c6d649e6fe5","Type":"ContainerStarted","Data":"a224647afab66b3dd230cd4bb5465c8f63d454a6b509c792c0a7fa0dcde13cac"}
Oct 30 18:46:24.886126 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.886107    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhb5w" event={"ID":"a768b8cc-62a2-4588-82df-edd68a625830","Type":"ContainerStarted","Data":"57f830ce9813a2765b4e791354df81b6eee6bb93ed858099b46d47435faa03f5"}
Oct 30 18:46:24.886218 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:24.886153    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhb5w" event={"ID":"a768b8cc-62a2-4588-82df-edd68a625830","Type":"ContainerStarted","Data":"10b4b395a3d09bf926d67e18f6550ba2d2b9e644522aabb9b8dc30e227ebce9f"}
Oct 30 18:46:25.763084 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.763047    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b9db974cb-slw8w"]
Oct 30 18:46:25.765508 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.765482    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.815832 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.815791    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lk2tt\""
Oct 30 18:46:25.815997 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.815976    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 18:46:25.816050 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.816037    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 18:46:25.816112 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.816072    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 18:46:25.816227 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.816213    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 18:46:25.816340 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.816320    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 18:46:25.816417 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.816388    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 18:46:25.816448 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.816428    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 18:46:25.828048 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.828015    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b9db974cb-slw8w"]
Oct 30 18:46:25.860690 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.860654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.860690 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.860688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.860959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.860742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.860959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.860767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.860959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.860782    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.860959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.860837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzb8k\" (UniqueName: \"kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.860959 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.860865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.895756 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.895711    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhb5w" event={"ID":"a768b8cc-62a2-4588-82df-edd68a625830","Type":"ContainerStarted","Data":"d3b9af6206f1def97f4d35c7a9afd3556d35a919b1a62ad30f6a0c5d673024de"}
Oct 30 18:46:25.961674 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.961634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.961874 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.961688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.961874 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.961716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.961874 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.961773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzb8k\" (UniqueName: \"kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.961874 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.961804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.961874 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.961866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.962114 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.961890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.962328 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.962310    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.962928 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.962904    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.964025 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.964000    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.964166 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.964032    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.965575 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.965555    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:25.966338 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:25.966312    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:26.020793 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.020715    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzb8k\" (UniqueName: \"kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:26.078982 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.078943    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:26.269483 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.269451    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"]
Oct 30 18:46:26.272729 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.272592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.321947 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.321906    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 18:46:26.322135 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.322017    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 18:46:26.322135 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.322106    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cntpx\""
Oct 30 18:46:26.322276 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.322148    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:46:26.322276 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.322218    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:46:26.322276 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.322251    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:46:26.332679 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.332647    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"]
Oct 30 18:46:26.365471 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.365180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3c313c3-122e-43ce-9d68-106881e7b7a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.365471 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.365244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.365471 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.365299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.365471 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.365327    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbhv5\" (UniqueName: \"kubernetes.io/projected/b3c313c3-122e-43ce-9d68-106881e7b7a5-kube-api-access-xbhv5\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.466001 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.465956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3c313c3-122e-43ce-9d68-106881e7b7a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.466186 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.466046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.466186 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.466122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.466186 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.466147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbhv5\" (UniqueName: \"kubernetes.io/projected/b3c313c3-122e-43ce-9d68-106881e7b7a5-kube-api-access-xbhv5\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.467303 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.466901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3c313c3-122e-43ce-9d68-106881e7b7a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.469606 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.469576    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.470131 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.470087    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.515955 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.515749    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b9db974cb-slw8w"]
Oct 30 18:46:26.523743 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:26.523677    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1853522_ac6d_49a5_b86b_d8056aadc5d5.slice/crio-6b153d5855fe6abf809b51f069addb0885ed65fca1333b0df2f85fccea4f461a WatchSource:0}: Error finding container 6b153d5855fe6abf809b51f069addb0885ed65fca1333b0df2f85fccea4f461a: Status 404 returned error can't find the container with id 6b153d5855fe6abf809b51f069addb0885ed65fca1333b0df2f85fccea4f461a
Oct 30 18:46:26.528042 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.528013    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbhv5\" (UniqueName: \"kubernetes.io/projected/b3c313c3-122e-43ce-9d68-106881e7b7a5-kube-api-access-xbhv5\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.588124 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.587914    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:26.900010 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.899977    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn7zd" event={"ID":"13af2e85-a46f-496a-9151-5c6d649e6fe5","Type":"ContainerStarted","Data":"690731fb40ba2cfe084a49a9a1233f34f4b25bba896a408f69d3d84a02e328fa"}
Oct 30 18:46:26.901352 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.901316    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9db974cb-slw8w" event={"ID":"b1853522-ac6d-49a5-b86b-d8056aadc5d5","Type":"ContainerStarted","Data":"6b153d5855fe6abf809b51f069addb0885ed65fca1333b0df2f85fccea4f461a"}
Oct 30 18:46:26.991997 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:26.991956    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"]
Oct 30 18:46:27.001023 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:27.000989    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c313c3_122e_43ce_9d68_106881e7b7a5.slice/crio-f023a1e938200e9c98d0cd0ee31a8a1aa178f625344ece225904f6b0b0d388dd WatchSource:0}: Error finding container f023a1e938200e9c98d0cd0ee31a8a1aa178f625344ece225904f6b0b0d388dd: Status 404 returned error can't find the container with id f023a1e938200e9c98d0cd0ee31a8a1aa178f625344ece225904f6b0b0d388dd
Oct 30 18:46:27.905717 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:27.905680    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tkr8v" event={"ID":"46605ee7-f737-431c-8a64-0b430a550d34","Type":"ContainerStarted","Data":"83fe5a75413e2ef1877fa755f825d8a6d291d5d8f28914629950bf1eda0d3098"}
Oct 30 18:46:27.907067 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:27.906993    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v" event={"ID":"b3c313c3-122e-43ce-9d68-106881e7b7a5","Type":"ContainerStarted","Data":"f023a1e938200e9c98d0cd0ee31a8a1aa178f625344ece225904f6b0b0d388dd"}
Oct 30 18:46:27.909211 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:27.909164    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn7zd" event={"ID":"13af2e85-a46f-496a-9151-5c6d649e6fe5","Type":"ContainerStarted","Data":"695c99f240e3a175b76776a61deb9139951eaba768426e4cfc31994b0bef2659"}
Oct 30 18:46:27.909396 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:27.909301    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:27.911533 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:27.911500    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhb5w" event={"ID":"a768b8cc-62a2-4588-82df-edd68a625830","Type":"ContainerStarted","Data":"c9aedfc9379329819a4b3b2deff5d405e20bfba6f8ab73ea55220f82243b4767"}
Oct 30 18:46:28.015836 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:28.015751    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tkr8v" podStartSLOduration=1.635067193 podStartE2EDuration="5.015734129s" totalImagesPullingTime="3.380666936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:28.015148509 +0000 UTC m=+53.918734109" watchObservedRunningTime="2026-10-30 18:46:28.015734129 +0000 UTC m=+53.919319729"
Oct 30 18:46:28.127015 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:28.126959    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zhb5w" podStartSLOduration=2.025578089 podStartE2EDuration="5.126945437s" totalImagesPullingTime="3.101367348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:46:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:28.126434725 +0000 UTC m=+54.030020335" watchObservedRunningTime="2026-10-30 18:46:28.126945437 +0000 UTC m=+54.030531030"
Oct 30 18:46:28.240614 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:28.240510    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-cn7zd" podStartSLOduration=2.723848674 podStartE2EDuration="5.240495967s" totalImagesPullingTime="2.516647293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:28.240043806 +0000 UTC m=+54.143629411" watchObservedRunningTime="2026-10-30 18:46:28.240495967 +0000 UTC m=+54.144081564"
Oct 30 18:46:28.916932 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:28.916735    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v" event={"ID":"b3c313c3-122e-43ce-9d68-106881e7b7a5","Type":"ContainerStarted","Data":"0ab22887049d488b5b37f6e49930eb4955d9dd19852ca4fcc1c9dc728a4377b6"}
Oct 30 18:46:29.841280 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:29.841255    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:29.926351 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:29.926305    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v" event={"ID":"b3c313c3-122e-43ce-9d68-106881e7b7a5","Type":"ContainerStarted","Data":"7b4324e30795b9946ca3692ffb799694ba4d967747bce77f6ee4e10dcde20859"}
Oct 30 18:46:30.185346 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:30.185253    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v" podStartSLOduration=2.349520149 podStartE2EDuration="4.185237945s" totalImagesPullingTime="1.835717796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:30.183947545 +0000 UTC m=+56.087533141" watchObservedRunningTime="2026-10-30 18:46:30.185237945 +0000 UTC m=+56.088823542"
Oct 30 18:46:32.937675 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:32.937640    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9db974cb-slw8w" event={"ID":"b1853522-ac6d-49a5-b86b-d8056aadc5d5","Type":"ContainerStarted","Data":"5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166"}
Oct 30 18:46:33.064631 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.064574    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7b9db974cb-slw8w" podStartSLOduration=2.648103778 podStartE2EDuration="8.064560248s" totalImagesPullingTime="5.41645647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:33.064101438 +0000 UTC m=+58.967687035" watchObservedRunningTime="2026-10-30 18:46:33.064560248 +0000 UTC m=+58.968145845"
Oct 30 18:46:33.110934 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.110896    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8jv7v"]
Oct 30 18:46:33.113323 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.113301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218457 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-root\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218457 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218620 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-sys\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218620 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-wtmp\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218620 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218579    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-tls\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218728 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218632    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-textfile\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218728 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-metrics-client-ca\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218884 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdv6f\" (UniqueName: \"kubernetes.io/projected/d22062d6-b0ae-4eed-acd4-f1142e650f41-kube-api-access-gdv6f\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218884 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.218989 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218886    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:46:33.218989 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.218955    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:46:33.219065 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.219025    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jrxvz\""
Oct 30 18:46:33.221595 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.221576    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:46:33.319916 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.319879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-textfile\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320078 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.319930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-metrics-client-ca\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320078 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdv6f\" (UniqueName: \"kubernetes.io/projected/d22062d6-b0ae-4eed-acd4-f1142e650f41-kube-api-access-gdv6f\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320259 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320259 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-root\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320259 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320259 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320192    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-root\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320259 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-sys\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320259 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320246    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-sys\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320491 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-wtmp\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320491 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320308    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-textfile\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320491 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-tls\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320491 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320445    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-wtmp\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320644 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320555    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-metrics-client-ca\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.320728 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.320708    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.322453 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.322433    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.322541 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.322525    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-tls\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.377358 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.377336    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdv6f\" (UniqueName: \"kubernetes.io/projected/d22062d6-b0ae-4eed-acd4-f1142e650f41-kube-api-access-gdv6f\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.426583 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.426534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:33.436185 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:33.436162    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22062d6_b0ae_4eed_acd4_f1142e650f41.slice/crio-ad9dd4899db49aa05f841cca6ffc25e45b50ddad0da464a1e4d292ecebb6ae86 WatchSource:0}: Error finding container ad9dd4899db49aa05f841cca6ffc25e45b50ddad0da464a1e4d292ecebb6ae86: Status 404 returned error can't find the container with id ad9dd4899db49aa05f841cca6ffc25e45b50ddad0da464a1e4d292ecebb6ae86
Oct 30 18:46:33.941438 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:33.941401    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jv7v" event={"ID":"d22062d6-b0ae-4eed-acd4-f1142e650f41","Type":"ContainerStarted","Data":"ad9dd4899db49aa05f841cca6ffc25e45b50ddad0da464a1e4d292ecebb6ae86"}
Oct 30 18:46:34.946412 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:34.946378    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jv7v" event={"ID":"d22062d6-b0ae-4eed-acd4-f1142e650f41","Type":"ContainerStarted","Data":"b24ee5d07e1133eba403c462e1dc64108e1845bca85be96175e5ee37e5db9e8c"}
Oct 30 18:46:35.950646 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:35.950607    2577 generic.go:356] "Generic (PLEG): container finished" podID="d22062d6-b0ae-4eed-acd4-f1142e650f41" containerID="b24ee5d07e1133eba403c462e1dc64108e1845bca85be96175e5ee37e5db9e8c" exitCode=0
Oct 30 18:46:35.951039 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:35.950669    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jv7v" event={"ID":"d22062d6-b0ae-4eed-acd4-f1142e650f41","Type":"ContainerDied","Data":"b24ee5d07e1133eba403c462e1dc64108e1845bca85be96175e5ee37e5db9e8c"}
Oct 30 18:46:35.951039 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:35.950760    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jv7v" event={"ID":"d22062d6-b0ae-4eed-acd4-f1142e650f41","Type":"ContainerStarted","Data":"4adec55c5a21aa94bef34fff99563ade0aa90e0fb00c80ab31ac69fb0e674947"}
Oct 30 18:46:35.951039 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:35.950786    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8jv7v" event={"ID":"d22062d6-b0ae-4eed-acd4-f1142e650f41","Type":"ContainerStarted","Data":"6eed9010d46d83c8010b5685b27fb027338bb5db6e3dfb7baf3a5d1b6459ebe3"}
Oct 30 18:46:36.070670 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:36.070618    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8jv7v" podStartSLOduration=1.630105674 podStartE2EDuration="3.070603699s" totalImagesPullingTime="1.374706034s" totalInitContainerRuntime="65.791991ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:36.069351775 +0000 UTC m=+61.972937373" watchObservedRunningTime="2026-10-30 18:46:36.070603699 +0000 UTC m=+61.974189296"
Oct 30 18:46:36.080103 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:36.080072    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:36.080221 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:36.080124    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:36.085019 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:36.084999    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:36.958372 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:36.958345    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:37.919061 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:37.919033    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:40.582503 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.582463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:40.582932 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.582527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:40.676580 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.676550    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:46:40.676720 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.676606    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:46:40.682852 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.682807    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:40.685079 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.685055    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:40.685177 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.685059    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:40.730757 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.730728    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:46:40.781556 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.781528    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:46:40.786394 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.786362    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:40.912235 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.912201    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgxcg\""
Oct 30 18:46:40.914591 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.914570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:40.917378 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.917363    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q9tbn\""
Oct 30 18:46:40.920527 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.920506    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:40.921184 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.921168    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-crxtc\""
Oct 30 18:46:40.924523 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:40.924502    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:41.305143 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:41.305051    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kml2f"]
Oct 30 18:46:41.348442 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:41.348412    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e3b2c8e_e1e8_499f_9c8b_164e47a57f54.slice/crio-f38a29303ef07383404c0b8ac3f391b516d2c777e07d3b70bc8377f87ead4879 WatchSource:0}: Error finding container f38a29303ef07383404c0b8ac3f391b516d2c777e07d3b70bc8377f87ead4879: Status 404 returned error can't find the container with id f38a29303ef07383404c0b8ac3f391b516d2c777e07d3b70bc8377f87ead4879
Oct 30 18:46:41.354209 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:41.354188    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29t9c"]
Oct 30 18:46:41.356453 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:41.356430    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f8whs"]
Oct 30 18:46:41.360677 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:41.360657    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod224cd297_364e_466b_a43e_3ac6a2de7d29.slice/crio-31c8f647c5cb783a4b9dd0c2165bd62c920ac9da68581886221c4945dd3a2f1f WatchSource:0}: Error finding container 31c8f647c5cb783a4b9dd0c2165bd62c920ac9da68581886221c4945dd3a2f1f: Status 404 returned error can't find the container with id 31c8f647c5cb783a4b9dd0c2165bd62c920ac9da68581886221c4945dd3a2f1f
Oct 30 18:46:41.363491 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:41.363467    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89987b8e_5b3f_4d20_b82e_643d32d7c004.slice/crio-3d435197f73cfcab16434af8c55a2916113b4e9d56ad0765f74b36f768649ee8 WatchSource:0}: Error finding container 3d435197f73cfcab16434af8c55a2916113b4e9d56ad0765f74b36f768649ee8: Status 404 returned error can't find the container with id 3d435197f73cfcab16434af8c55a2916113b4e9d56ad0765f74b36f768649ee8
Oct 30 18:46:41.971031 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:41.970993    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29t9c" event={"ID":"224cd297-364e-466b-a43e-3ac6a2de7d29","Type":"ContainerStarted","Data":"31c8f647c5cb783a4b9dd0c2165bd62c920ac9da68581886221c4945dd3a2f1f"}
Oct 30 18:46:41.972099 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:41.972074    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kml2f" event={"ID":"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54","Type":"ContainerStarted","Data":"f38a29303ef07383404c0b8ac3f391b516d2c777e07d3b70bc8377f87ead4879"}
Oct 30 18:46:41.973162 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:41.973143    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f8whs" event={"ID":"89987b8e-5b3f-4d20-b82e-643d32d7c004","Type":"ContainerStarted","Data":"3d435197f73cfcab16434af8c55a2916113b4e9d56ad0765f74b36f768649ee8"}
Oct 30 18:46:42.132552 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.132517    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:46:42.135750 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.135728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.187216 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.187186    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 18:46:42.187394 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.187368    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 18:46:42.187445 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.187384    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 18:46:42.187492 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.187450    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 18:46:42.187633 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.187615    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 18:46:42.187748 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.187735    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6zff9\""
Oct 30 18:46:42.187798 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.187783    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195577    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195709    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196304 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.195972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.196021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.196077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.196105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.196897 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.196150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxpvg\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.233690 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.233597    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 18:46:42.233690 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.233605    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7tn3b1m8racr8\""
Oct 30 18:46:42.280380 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.280349    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 18:46:42.282215 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.282190    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:46:42.285547 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.285518    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 18:46:42.285547 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.285537    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 18:46:42.285677 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.285557    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 18:46:42.287687 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.287668    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:46:42.287782 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.287771    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 18:46:42.288045 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.288032    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 18:46:42.297330 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.297306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.297440 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.297340    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.297440 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.297368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxpvg\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298090 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298070    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298180 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298180 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298180 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298180 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298338    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298368 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298768 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298768 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298894 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.298894 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.298863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.301029 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.299840    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.301029 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.299981    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.307054 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.303219    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.307054 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.303258    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.307054 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.303272    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.307054 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.304576    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.309242 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.309217    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.311342 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.311317    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.320021 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.319996    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.320212 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.320188    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.320278 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.320237    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.320408 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.320380    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.320507 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.320490    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.320550 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.319401    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.320765 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.320746    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.321050 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.321025    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.321206 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.321160    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.326667 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.326642    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 18:46:42.331311 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.331288    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.385532 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.385501    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxpvg\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.457549 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.457494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:42.899880 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.899845    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:46:42.910839 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:42.910300    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4ac65f_7a27_48a5_9bce_1fe44944d664.slice/crio-737ccf945120bedf59271c77ac49cd073328e7ed96c155244050bb4edca5d3a9 WatchSource:0}: Error finding container 737ccf945120bedf59271c77ac49cd073328e7ed96c155244050bb4edca5d3a9: Status 404 returned error can't find the container with id 737ccf945120bedf59271c77ac49cd073328e7ed96c155244050bb4edca5d3a9
Oct 30 18:46:42.978634 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:42.978597    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerStarted","Data":"737ccf945120bedf59271c77ac49cd073328e7ed96c155244050bb4edca5d3a9"}
Oct 30 18:46:43.986338 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:43.985556    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29t9c" event={"ID":"224cd297-364e-466b-a43e-3ac6a2de7d29","Type":"ContainerStarted","Data":"e2b8dab37bcd64b8c119fef528fd25f2fab52a02fbdade199257a66e1376a8f6"}
Oct 30 18:46:43.986338 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:43.985624    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29t9c" event={"ID":"224cd297-364e-466b-a43e-3ac6a2de7d29","Type":"ContainerStarted","Data":"79548d546bafc867fc627b4ab0590f149a9cc793bf5825cfd3bbed127bb9da7e"}
Oct 30 18:46:44.097122 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:44.097058    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-29t9c" podStartSLOduration=67.276289698 podStartE2EDuration="1m9.09703905s" totalImagesPullingTime="1.820749352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:44.096751222 +0000 UTC m=+70.000336820" watchObservedRunningTime="2026-10-30 18:46:44.09703905 +0000 UTC m=+70.000624648"
Oct 30 18:46:44.991932 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:44.991900    2577 generic.go:356] "Generic (PLEG): container finished" podID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038" exitCode=0
Oct 30 18:46:44.992400 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:44.992025    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerDied","Data":"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"}
Oct 30 18:46:45.478061 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.478036    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f888ff8d9-78xgg"]
Oct 30 18:46:45.486448 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.486427    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.541576 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.541547    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 18:46:45.545847 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.545802    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f888ff8d9-78xgg"]
Oct 30 18:46:45.629499 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.629464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.629642 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.629506    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5nrq\" (UniqueName: \"kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.629642 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.629597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.629642 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.629635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.629780 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.629663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.629837 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.629804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.629893 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.629873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.629946 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.629893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.730566 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.730478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.730566 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.730529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.730782 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.730647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5nrq\" (UniqueName: \"kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.730782 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.730705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.730782 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.730734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.730782 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.730771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.731014 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.730829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.731014 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.730886    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.731348 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.731321    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.731557 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.731538    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.731630 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.731611    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.731685 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.731632    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.734476 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.734455    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.735283 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.735237    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.744633 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.744604    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.789962 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.789928    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5nrq\" (UniqueName: \"kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.798628 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.798596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:45.997438 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.997351    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f8whs" event={"ID":"89987b8e-5b3f-4d20-b82e-643d32d7c004","Type":"ContainerStarted","Data":"1a75b74d34ec5717721df76ab1b8da48f3d719537ba2fda2b6cba41647492159"}
Oct 30 18:46:45.997862 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:45.997490    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:46.106935 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:46.106873    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f8whs" podStartSLOduration=68.052203235 podStartE2EDuration="1m12.106853409s" totalImagesPullingTime="4.054650174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:46.105874147 +0000 UTC m=+72.009459745" watchObservedRunningTime="2026-10-30 18:46:46.106853409 +0000 UTC m=+72.010439008"
Oct 30 18:46:46.233535 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:46.233449    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f888ff8d9-78xgg"]
Oct 30 18:46:46.241586 ip-10-0-143-248 kubenswrapper[2577]: W1030 18:46:46.241541    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731eee3b_34bd_4188_9c74_a6a684acce6b.slice/crio-b8c05cd7abffae36b8d4f9dbfc12e164f19673eb786383f1486e38c2f25feb0b WatchSource:0}: Error finding container b8c05cd7abffae36b8d4f9dbfc12e164f19673eb786383f1486e38c2f25feb0b: Status 404 returned error can't find the container with id b8c05cd7abffae36b8d4f9dbfc12e164f19673eb786383f1486e38c2f25feb0b
Oct 30 18:46:47.002570 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:47.002534    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f888ff8d9-78xgg" event={"ID":"731eee3b-34bd-4188-9c74-a6a684acce6b","Type":"ContainerStarted","Data":"75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e"}
Oct 30 18:46:47.003018 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:47.002597    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f888ff8d9-78xgg" event={"ID":"731eee3b-34bd-4188-9c74-a6a684acce6b","Type":"ContainerStarted","Data":"b8c05cd7abffae36b8d4f9dbfc12e164f19673eb786383f1486e38c2f25feb0b"}
Oct 30 18:46:47.119222 ip-10-0-143-248 kubenswrapper[2577]: I1030 18:46:47.119170    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5f888ff8d9-78xgg" podStartSLOduration=2.119154493 podStartE2EDuration="2.119154493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:47.116408654 +0000 UTC m=+73.019994251" watchObservedRunningTime="2026-10-30 18:46:47.119154493 +0000 UTC m=+73.022740163"
Oct 30 18:46:47.360117 ip-10-0-143-248 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 18:46:47.374297 ip-10-0-143-248 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 18:46:47.374526 ip-10-0-143-248 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 18:46:47.374592 ip-10-0-143-248 systemd[1]: kubelet.service: Consumed 3.884s CPU time.
Oct 30 18:46:47.384571 ip-10-0-143-248 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:46:47.444159 ip-10-0-143-248 kubenswrapper[5232]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:46:47.444159 ip-10-0-143-248 kubenswrapper[5232]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:46:47.444159 ip-10-0-143-248 kubenswrapper[5232]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:46:47.444159 ip-10-0-143-248 kubenswrapper[5232]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:46:47.446504 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446423    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:46:47.446504 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446503    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:46:47.446504 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446506    5232 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:46:47.446504 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446509    5232 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446512    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446515    5232 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446519    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446522    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446524    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446527    5232 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446530    5232 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446532    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446536    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446538    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446541    5232 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446544    5232 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446549    5232 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446553    5232 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446559    5232 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446563    5232 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446566    5232 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446569    5232 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446571    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:46:47.446652 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446574    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446577    5232 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446585    5232 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446588    5232 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446591    5232 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446593    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446596    5232 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446600    5232 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446602    5232 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446605    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446608    5232 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446611    5232 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446614    5232 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446617    5232 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446619    5232 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446622    5232 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446625    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446627    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446630    5232 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:46:47.447140 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446632    5232 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446635    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446638    5232 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446641    5232 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446644    5232 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446646    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446649    5232 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446651    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446654    5232 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446659    5232 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446664    5232 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446668    5232 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446671    5232 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446674    5232 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446676    5232 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446679    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446682    5232 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446684    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446687    5232 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446689    5232 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:46:47.447595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446692    5232 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446694    5232 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446697    5232 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446699    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446702    5232 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446707    5232 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446711    5232 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446714    5232 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446716    5232 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446719    5232 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446726    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446729    5232 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446731    5232 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446734    5232 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446737    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446739    5232 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446742    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446745    5232 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446747    5232 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:46:47.448090 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446750    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446753    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446756    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446759    5232 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446761    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446764    5232 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446767    5232 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446769    5232 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446772    5232 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446774    5232 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446777    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446780    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446782    5232 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446785    5232 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446787    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446790    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446795    5232 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446798    5232 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446801    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:46:47.448544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446804    5232 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446806    5232 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446829    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446832    5232 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446837    5232 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.446840    5232 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447603    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447610    5232 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447613    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447617    5232 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447621    5232 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447637    5232 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447641    5232 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447645    5232 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447648    5232 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447651    5232 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447654    5232 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447657    5232 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447659    5232 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447662    5232 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:46:47.449043 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447665    5232 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447667    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447670    5232 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447673    5232 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447675    5232 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447678    5232 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447680    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447683    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447685    5232 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447688    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447691    5232 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447693    5232 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447697    5232 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447702    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447705    5232 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447709    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447716    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447720    5232 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447725    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447729    5232 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:46:47.449520 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447732    5232 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447735    5232 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447738    5232 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447741    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447744    5232 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447747    5232 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447749    5232 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447752    5232 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447756    5232 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447760    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447763    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447765    5232 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447768    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447771    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447773    5232 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447776    5232 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447779    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447781    5232 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447784    5232 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:46:47.450016 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447786    5232 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447789    5232 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447792    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447795    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447798    5232 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447801    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447804    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447806    5232 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447823    5232 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447827    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447830    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447833    5232 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447836    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447839    5232 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447841    5232 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447844    5232 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447846    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447849    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447851    5232 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447854    5232 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:46:47.450517 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447856    5232 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447859    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447861    5232 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447864    5232 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447866    5232 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447869    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447880    5232 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447883    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447886    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447889    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447891    5232 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447894    5232 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447897    5232 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447899    5232 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447903    5232 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447907    5232 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447911    5232 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447914    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447917    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:46:47.451282 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447920    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447923    5232 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447927    5232 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447929    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447932    5232 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447934    5232 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447938    5232 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447940    5232 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447943    5232 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447946    5232 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447948    5232 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447951    5232 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447953    5232 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.447957    5232 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448426    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448431    5232 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448434    5232 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448437    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448439    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448442    5232 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:46:47.452044 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448445    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448447    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448450    5232 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448453    5232 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448455    5232 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448458    5232 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448461    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448464    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448466    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448469    5232 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448472    5232 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448475    5232 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448477    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448480    5232 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448484    5232 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448488    5232 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448491    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448494    5232 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448497    5232 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:46:47.452769 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448500    5232 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448502    5232 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448505    5232 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448507    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448510    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448512    5232 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448515    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448517    5232 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448520    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448522    5232 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448525    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448527    5232 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448530    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448532    5232 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448535    5232 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448539    5232 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448542    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448544    5232 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448547    5232 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448550    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:46:47.453312 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448553    5232 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448556    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448558    5232 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448561    5232 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448564    5232 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448566    5232 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448569    5232 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448571    5232 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448574    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448577    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448579    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448582    5232 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448585    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448587    5232 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448590    5232 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448593    5232 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448595    5232 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448598    5232 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448600    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448603    5232 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:46:47.453798 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448605    5232 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448610    5232 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448614    5232 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448617    5232 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448620    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448623    5232 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448626    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448630    5232 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448632    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448635    5232 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448637    5232 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448640    5232 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448643    5232 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448645    5232 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448648    5232 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448650    5232 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448653    5232 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448655    5232 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448658    5232 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:46:47.454318 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448660    5232 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448663    5232 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448665    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448668    5232 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448671    5232 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448673    5232 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448676    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448679    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448681    5232 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448684    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448686    5232 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448689    5232 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448691    5232 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448695    5232 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448697    5232 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448700    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448702    5232 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448705    5232 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448707    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448709    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:46:47.455259 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448713    5232 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.448716    5232 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448822    5232 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448836    5232 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448843    5232 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448847    5232 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448851    5232 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448855    5232 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448859    5232 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448863    5232 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448866    5232 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448870    5232 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448873    5232 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448877    5232 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448880    5232 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448883    5232 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448886    5232 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448889    5232 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448892    5232 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448896    5232 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448903    5232 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448906    5232 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448909    5232 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448912    5232 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:46:47.456169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448915    5232 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448919    5232 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448922    5232 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448926    5232 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448929    5232 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448932    5232 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448935    5232 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448938    5232 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448943    5232 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448946    5232 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448951    5232 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448954    5232 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448957    5232 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448960    5232 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448963    5232 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448966    5232 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448971    5232 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448974    5232 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448977    5232 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448980    5232 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448983    5232 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448987    5232 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448991    5232 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448994    5232 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.448997    5232 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:46:47.457237 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449000    5232 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449003    5232 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449006    5232 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449011    5232 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449014    5232 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449017    5232 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449020    5232 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449024    5232 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449027    5232 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449030    5232 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449033    5232 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449037    5232 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449040    5232 flags.go:64] FLAG: --help="false"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449043    5232 flags.go:64] FLAG: --hostname-override="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449046    5232 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449051    5232 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449054    5232 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449058    5232 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449061    5232 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449064    5232 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449067    5232 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449070    5232 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449073    5232 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:46:47.458384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449077    5232 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449080    5232 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449083    5232 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449086    5232 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449089    5232 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449092    5232 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449095    5232 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449098    5232 flags.go:64] FLAG: --lock-file=""
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449101    5232 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449105    5232 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449107    5232 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449113    5232 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449117    5232 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449120    5232 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449123    5232 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449127    5232 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449130    5232 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449133    5232 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449136    5232 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449141    5232 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449144    5232 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449148    5232 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449151    5232 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449154    5232 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449159    5232 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:46:47.459394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449162    5232 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449165    5232 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449168    5232 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449171    5232 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449178    5232 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449181    5232 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449184    5232 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449187    5232 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449190    5232 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449193    5232 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449196    5232 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449199    5232 flags.go:64] FLAG: --port="10250"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449203    5232 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449205    5232 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0eb2697fdb173e241"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449209    5232 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449212    5232 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449215    5232 flags.go:64] FLAG: --register-node="true"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449217    5232 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449221    5232 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449225    5232 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449228    5232 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449232    5232 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449236    5232 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449239    5232 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449242    5232 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449245    5232 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:46:47.460428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449248    5232 flags.go:64] FLAG: --runonce="false"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449251    5232 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449254    5232 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449257    5232 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449260    5232 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449265    5232 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449269    5232 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449272    5232 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449275    5232 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449278    5232 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449281    5232 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449284    5232 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449286    5232 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449289    5232 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449292    5232 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449295    5232 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449301    5232 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449303    5232 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449306    5232 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449311    5232 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449314    5232 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449317    5232 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449320    5232 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449323    5232 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449326    5232 flags.go:64] FLAG: --v="2"
Oct 30 18:46:47.461317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449331    5232 flags.go:64] FLAG: --version="false"
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449335    5232 flags.go:64] FLAG: --vmodule=""
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449339    5232 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449344    5232 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449506    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449512    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449515    5232 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449518    5232 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449521    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449524    5232 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449527    5232 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449530    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449534    5232 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449537    5232 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449540    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449543    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449545    5232 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449548    5232 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449550    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449553    5232 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:46:47.461985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449557    5232 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449560    5232 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449564    5232 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449569    5232 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449573    5232 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449577    5232 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449580    5232 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449584    5232 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449588    5232 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449594    5232 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449599    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449603    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449609    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449613    5232 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449616    5232 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449620    5232 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449624    5232 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449628    5232 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449632    5232 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:46:47.462474 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449636    5232 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449640    5232 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449644    5232 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449647    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449652    5232 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449657    5232 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449662    5232 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449666    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449670    5232 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449673    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449685    5232 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449689    5232 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449693    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449697    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449701    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449705    5232 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449709    5232 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449713    5232 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449717    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449722    5232 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:46:47.462949 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449726    5232 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449730    5232 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449735    5232 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449739    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449743    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449750    5232 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449755    5232 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449761    5232 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449767    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449771    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449782    5232 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449786    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449788    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449792    5232 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449794    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449797    5232 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449800    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449805    5232 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449807    5232 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:46:47.463448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449825    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449829    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449832    5232 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449834    5232 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449837    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449840    5232 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449843    5232 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449846    5232 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449848    5232 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449851    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449853    5232 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449856    5232 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449859    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449861    5232 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449864    5232 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449866    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449869    5232 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449872    5232 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449876    5232 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:46:47.464052 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449879    5232 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449881    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449884    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449888    5232 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449893    5232 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449905    5232 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449910    5232 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449913    5232 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449916    5232 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449919    5232 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449921    5232 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449925    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.449928    5232 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.449964    5232 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450016    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450023    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:46:47.464544 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450029    5232 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450034    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450038    5232 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450042    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450046    5232 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450052    5232 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450056    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450060    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450064    5232 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450068    5232 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450072    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450077    5232 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450081    5232 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450085    5232 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450089    5232 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450093    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450100    5232 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450106    5232 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450110    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:46:47.465032 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450114    5232 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450119    5232 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450123    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450137    5232 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450143    5232 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450148    5232 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450153    5232 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450158    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450164    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450169    5232 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450174    5232 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450178    5232 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450182    5232 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450185    5232 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450188    5232 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450191    5232 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450193    5232 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450196    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450199    5232 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450202    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:46:47.465477 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450205    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450207    5232 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450210    5232 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450214    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450218    5232 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450222    5232 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450226    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450230    5232 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450235    5232 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450242    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450245    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450248    5232 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450250    5232 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450253    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450255    5232 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450258    5232 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450268    5232 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450271    5232 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450274    5232 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450278    5232 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:46:47.466094 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450285    5232 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450289    5232 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450294    5232 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450299    5232 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450303    5232 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450308    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450311    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450315    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450318    5232 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450320    5232 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450323    5232 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450326    5232 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450328    5232 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450331    5232 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450334    5232 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450337    5232 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450339    5232 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450342    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450345    5232 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:46:47.466616 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450347    5232 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450350    5232 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450354    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450357    5232 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450359    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450362    5232 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450365    5232 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450368    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450370    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450373    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450381    5232 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450384    5232 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450387    5232 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450391    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450394    5232 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450397    5232 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450400    5232 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450402    5232 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450405    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450408    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:46:47.467123 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450411    5232 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450413    5232 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450418    5232 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450422    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450425    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.450429    5232 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.457175    5232 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.457196    5232 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.457223    5232 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.457230    5232 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457320    5232 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457328    5232 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457335    5232 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457340    5232 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457345    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457350    5232 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457354    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457358    5232 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457363    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:46:47.467595 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457367    5232 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457371    5232 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457377    5232 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457382    5232 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457387    5232 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457391    5232 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457396    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457401    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457406    5232 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457411    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457415    5232 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457419    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457424    5232 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457428    5232 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457432    5232 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457437    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457441    5232 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457445    5232 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457450    5232 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457455    5232 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:46:47.468069 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457459    5232 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457464    5232 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457468    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457473    5232 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457477    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457481    5232 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457486    5232 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457490    5232 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457494    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457498    5232 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457503    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457507    5232 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457511    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457516    5232 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457520    5232 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457525    5232 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457531    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457535    5232 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457540    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457544    5232 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:46:47.468551 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457548    5232 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457553    5232 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457557    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457561    5232 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457566    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457570    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457574    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457578    5232 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457583    5232 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457587    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457592    5232 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457597    5232 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457601    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457605    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457609    5232 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457615    5232 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457620    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457628    5232 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457634    5232 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:46:47.469144 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457641    5232 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457645    5232 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457650    5232 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457654    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457658    5232 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457663    5232 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457668    5232 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457672    5232 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457676    5232 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457681    5232 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457686    5232 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457691    5232 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457696    5232 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457700    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457704    5232 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457709    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457713    5232 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457718    5232 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457722    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457727    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:46:47.469593 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457731    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457735    5232 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457739    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457744    5232 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457748    5232 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457753    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457757    5232 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457762    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457766    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457770    5232 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457775    5232 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457779    5232 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457785    5232 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457792    5232 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457797    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457801    5232 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457806    5232 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:46:47.470072 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.457825    5232 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.457870    5232 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458106    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458114    5232 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458119    5232 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458123    5232 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458129    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458133    5232 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458137    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458142    5232 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458146    5232 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458150    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458155    5232 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458159    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458163    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458168    5232 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:46:47.470472 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458172    5232 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458177    5232 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458181    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458185    5232 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458190    5232 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458194    5232 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458199    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458203    5232 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458208    5232 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458212    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458216    5232 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458220    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458225    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458229    5232 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458234    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458238    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458242    5232 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458246    5232 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458251    5232 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458255    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:46:47.470911 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458259    5232 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458263    5232 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458268    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458274    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458278    5232 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458283    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458287    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458291    5232 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458296    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458300    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458304    5232 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458308    5232 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458313    5232 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458317    5232 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458321    5232 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458326    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458331    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458335    5232 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458339    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:46:47.471381 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458343    5232 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458348    5232 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458352    5232 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458356    5232 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458361    5232 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458365    5232 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458370    5232 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458374    5232 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458378    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458383    5232 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458387    5232 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458390    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458394    5232 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458398    5232 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458402    5232 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458406    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458410    5232 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458415    5232 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458419    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458424    5232 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:46:47.471832 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458429    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458433    5232 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458439    5232 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458445    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458450    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458455    5232 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458459    5232 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458463    5232 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458468    5232 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458473    5232 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458477    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458482    5232 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458486    5232 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458491    5232 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458497    5232 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458503    5232 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458507    5232 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458512    5232 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458516    5232 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:46:47.472307 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458521    5232 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458526    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458530    5232 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458535    5232 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458539    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458543    5232 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458547    5232 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458552    5232 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458556    5232 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458561    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458565    5232 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458570    5232 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458575    5232 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458579    5232 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.458617    5232 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:46:47.472751 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458674    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458680    5232 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458684    5232 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458689    5232 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458695    5232 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458727    5232 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458734    5232 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458739    5232 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458743    5232 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458748    5232 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458753    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458757    5232 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458762    5232 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458766    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458771    5232 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458775    5232 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458779    5232 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458784    5232 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458788    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:46:47.473159 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458792    5232 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458796    5232 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458801    5232 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458806    5232 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458826    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458830    5232 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458835    5232 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458839    5232 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458844    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458848    5232 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458853    5232 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458857    5232 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458862    5232 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458868    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458873    5232 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458877    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458881    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458885    5232 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458890    5232 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458895    5232 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:46:47.473606 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458899    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458904    5232 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458908    5232 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458912    5232 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458917    5232 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458921    5232 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458926    5232 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458931    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458935    5232 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458939    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458944    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458948    5232 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458952    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458957    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458961    5232 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458965    5232 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458970    5232 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458974    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458979    5232 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458983    5232 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:46:47.474091 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458988    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458992    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.458996    5232 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459001    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459006    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459012    5232 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459016    5232 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459020    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459024    5232 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459029    5232 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459033    5232 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459038    5232 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459042    5232 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459046    5232 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459051    5232 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459055    5232 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459059    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459064    5232 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459068    5232 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:46:47.474562 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459072    5232 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459077    5232 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459082    5232 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459086    5232 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459090    5232 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459095    5232 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459099    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459105    5232 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459109    5232 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459113    5232 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459117    5232 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459122    5232 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459126    5232 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459131    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459135    5232 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459139    5232 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459144    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459149    5232 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459155    5232 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:46:47.475020 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459160    5232 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459168    5232 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459175    5232 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459181    5232 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459186    5232 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459190    5232 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459194    5232 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459198    5232 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:46:47.459203    5232 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.459275    5232 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.461429    5232 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.461489    5232 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.461677    5232 server.go:1014] "Starting client certificate rotation"
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.461824    5232 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.461868    5232 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:40:34 +0000 UTC" deadline="2028-05-21 23:44:24.648870759 +0000 UTC"
Oct 30 18:46:47.475537 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.461894    5232 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13660h57m37.186978571s"
Oct 30 18:46:47.476276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.462159    5232 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:46:47.476276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.463202    5232 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:46:47.476276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.464316    5232 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:46:47.476276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.465982    5232 log.go:25] "Validated CRI v1 image API"
Oct 30 18:46:47.476276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.466627    5232 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:46:47.476276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.468437    5232 fs.go:135] Filesystem UUIDs: map[6e667742-f335-46c7-b2a5-ab43ac269d5b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d25e609b-5ef7-4b07-bc51-17e2a17f381b:/dev/nvme0n1p4]
Oct 30 18:46:47.476545 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.468458    5232 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0267d1961df22fac98e440ea5167ed61574d1fb617f5817dd360955ee2202b74/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0267d1961df22fac98e440ea5167ed61574d1fb617f5817dd360955ee2202b74/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/09b151b0942db019c719857b6ffef310a633086971628d385da08d8345221265/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/09b151b0942db019c719857b6ffef310a633086971628d385da08d8345221265/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/10b4b395a3d09bf926d67e18f6550ba2d2b9e644522aabb9b8dc30e227ebce9f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/10b4b395a3d09bf926d67e18f6550ba2d2b9e644522aabb9b8dc30e227ebce9f/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/31c8f647c5cb783a4b9dd0c2165bd62c920ac9da68581886221c4945dd3a2f1f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/31c8f647c5cb783a4b9dd0c2165bd62c920ac9da68581886221c4945dd3a2f1f/userdata/shm major:0 minor:301 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3d435197f73cfcab16434af8c55a2916113b4e9d56ad0765f74b36f768649ee8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3d435197f73cfcab16434af8c55a2916113b4e9d56ad0765f74b36f768649ee8/userdata/shm major:0 minor:304 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6b153d5855fe6abf809b51f069addb0885ed65fca1333b0df2f85fccea4f461a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6b153d5855fe6abf809b51f069addb0885ed65fca1333b0df2f85fccea4f461a/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/737ccf945120bedf59271c77ac49cd073328e7ed96c155244050bb4edca5d3a9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/737ccf945120bedf59271c77ac49cd073328e7ed96c155244050bb4edca5d3a9/userdata/shm major:0 minor:325 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8cb7324d074e007fe1f25f559bfd510fdf3c214b75716ffdc895dc48bac439c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8cb7324d074e007fe1f25f559bfd510fdf3c214b75716ffdc895dc48bac439c5/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8f30f6336b2dbfa9f8ab1c3ebbf735e06d291b8723261dd9aea1515c83f7c308/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8f30f6336b2dbfa9f8ab1c3ebbf735e06d291b8723261dd9aea1515c83f7c308/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9398e679b4eaef3f2b9f8d16b4395486a09a9ecc85e2e2c21edb803c0cde8ad5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9398e679b4eaef3f2b9f8d16b4395486a09a9ecc85e2e2c21edb803c0cde8ad5/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a224647afab66b3dd230cd4bb5465c8f63d454a6b509c792c0a7fa0dcde13cac/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a224647afab66b3dd230cd4bb5465c8f63d454a6b509c792c0a7fa0dcde13cac/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a92d5376dba793ee1a873662d6b3cfe8968b2fd4ad2177a12f54e3a4587332a5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a92d5376dba793ee1a873662d6b3cfe8968b2fd4ad2177a12f54e3a4587332a5/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ad9dd4899db49aa05f841cca6ffc25e45b50ddad0da464a1e4d292ecebb6ae86/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ad9dd4899db49aa05f841cca6ffc25e45b50ddad0da464a1e4d292ecebb6ae86/userdata/shm major:0 minor:284 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b8c05cd7abffae36b8d4f9dbfc12e164f19673eb786383f1486e38c2f25feb0b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b8c05cd7abffae36b8d4f9dbfc12e164f19673eb786383f1486e38c2f25feb0b/userdata/shm major:0 minor:349 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c12213cefbad597648aca4a0b3a428db72476e55a88d8df3bcc206128326cbc4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c12213cefbad597648aca4a0b3a428db72476e55a88d8df3bcc206128326cbc4/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c1f497db70bfc0bd39ee6ff3870b5fdc586c37138c25460d59bc5cc5ae1cdd0c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c1f497db70bfc0bd39ee6ff3870b5fdc586c37138c25460d59bc5cc5ae1cdd0c/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d417883d4c4541976655c44ca1b09639ea9e465b723673cd5b7da7596a6c76d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d417883d4c4541976655c44ca1b09639ea9e465b723673cd5b7da7596a6c76d5/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eda69974865db84652f4b9fe0fb6d591f9453fb23888e4d332d3a7b7bce95944/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eda69974865db84652f4b9fe0fb6d591f9453fb23888e4d332d3a7b7bce95944/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f023a1e938200e9c98d0cd0ee31a8a1aa178f625344ece225904f6b0b0d388dd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f023a1e938200e9c98d0cd0ee31a8a1aa178f625344ece225904f6b0b0d388dd/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f38a29303ef07383404c0b8ac3f391b516d2c777e07d3b70bc8377f87ead4879/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f38a29303ef07383404c0b8ac3f391b516d2c777e07d3b70bc8377f87ead4879/userdata/shm major:0 minor:302 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb81cc9b032d077ea698198cdb7a5af6674499023cce89e4cecdddc1341b2886/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb81cc9b032d077ea698198cdb7a5af6674499023cce89e4cecdddc1341b2886/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0dce46b5-9bcf-4b26-8126-cac8319d7a9a/volumes/kubernetes.io~projected/kube-api-access-8m4sr:{mountpoint:/var/lib/kubelet/pods/0dce46b5-9bcf-4b26-8126-cac8319d7a9a/volumes/kubernetes.io~projected/kube-api-access-8m4sr major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13af2e85-a46f-496a-9151-5c6d649e6fe5/volumes/kubernetes.io~projected/kube-api-access-jjq6z:{mountpoint:/var/lib/kubelet/pods/13af2e85-a46f-496a-9151-5c6d649e6fe5/volumes/kubernetes.io~projected/kube-api-access-jjq6z major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13af2e85-a46f-496a-9151-5c6d649e6fe5/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/13af2e85-a46f-496a-9151-5c6d649e6fe5/volumes/kubernetes.io~secret/metrics-tls major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volumes/kubernetes.io~projected/kube-api-access-fq9pz:{mountpoint:/var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volumes/kubernetes.io~projected/kube-api-access-fq9pz major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20a81c77-50d1-4e5b-a59d-43d455fb842d/volumes/kubernetes.io~projected/kube-api-access-m82st:{mountpoint:/var/lib/kubelet/pods/20a81c77-50d1-4e5b-a59d-43d455fb842d/volumes/kubernetes.io~projected/kube-api-access-m82st major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/224cd297-364e-466b-a43e-3ac6a2de7d29/volumes/kubernetes.io~projected/kube-api-access-bhcnv:{mountpoint:/var/lib/kubelet/pods/224cd297-364e-466b-a43e-3ac6a2de7d29/volumes/kubernetes.io~projected/kube-api-access-bhcnv major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/224cd297-364e-466b-a43e-3ac6a2de7d29/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/224cd297-364e-466b-a43e-3ac6a2de7d29/volumes/kubernetes.io~secret/metrics-certs major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46605ee7-f737-431c-8a64-0b430a550d34/volumes/kubernetes.io~projected/kube-api-access-j58xm:{mountpoint:/var/lib/kubelet/pods/46605ee7-f737-431c-8a64-0b430a550d34/volumes/kubernetes.io~projected/kube-api-access-j58xm major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46605ee7-f737-431c-8a64-0b430a550d34/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/46605ee7-f737-431c-8a64-0b430a550d34/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~empty-dir/config-out major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~projected/kube-api-access-rxpvg:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~projected/kube-api-access-rxpvg major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~projected/tls-assets major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/config major:0 minor:317 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:316 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/web-config major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~projected/kube-api-access-d5nrq:{mountpoint:/var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~projected/kube-api-access-d5nrq major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~projected/kube-api-access-vh6dp:{mountpoint:/var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~projected/kube-api-access-vh6dp major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/89987b8e-5b3f-4d20-b82e-643d32d7c004/volumes/kubernetes.io~projected/kube-api-access-74qsz:{mountpoint:/var/lib/kubelet/pods/89987b8e-5b3f-4d20-b82e-643d32d7c004/volumes/kubernetes.io~projected/kube-api-access-74qsz major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/96a93e14-0292-45a9-8e4b-b5a130ed3d55/volumes/kubernetes.io~projected/kube-api-access-2gmll:{mountpoint:/var/lib/kubelet/pods/96a93e14-0292-45a9-8e4b-b5a130ed3d55/volumes/kubernetes.io~projected/kube-api-access-2gmll major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a768b8cc-62a2-4588-82df-edd68a625830/volumes/kubernetes.io~projected/kube-api-access-nqn7d:{mountpoint:/var/lib/kubelet/pods/a768b8cc-62a2-4588-82df-edd68a625830/volumes/kubernetes.io~projected/kube-api-access-nqn7d major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a768b8cc-62a2-4588-82df-edd68a625830/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/a768b8cc-62a2-4588-82df-edd68a625830/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~projected/kube-api-access-wzb8k:{mountpoint:/var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~projected/kube-api-access-wzb8k major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b202127e-afab-4cac-bee8-e93533bc25d5/volumes/kubernetes.io~projected/kube-api-access-n4b4m:{mountpoint:/var/lib/kubelet/pods/b202127e-afab-4cac-bee8-e93533bc25d5/volumes/kubernetes.io~projected/kube-api-access-n4b4m major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~projected/kube-api-access-xbhv5:{mountpoint:/var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~projected/kube-api-access-xbhv5 major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4075d5d-2d2d-4e0e-b913-12715719e711/volumes/kubernetes.io~projected/kube-api-access-kppzz:{mountpoint:/var/lib/kubelet/pods/b4075d5d-2d2d-4e0e-b913-12715719e711/volumes/kubernetes.io~projected/kube-api-access-kppzz major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbeaac75-73d1-4d21-ad63-7215bb75e8e6/volumes/kubernetes.io~projected/kube-api-access-zwnlq:{mountpoint:/var/lib/kubelet/pods/bbeaac75-73d1-4d21-ad63-7215bb75e8e6/volumes/kubernetes.io~projected/kube-api-access-zwnlq major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~projected/kube-api-access-gdv6f:{mountpoint:/var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~projected/kube-api-access-gdv6f major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:282 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/24126a7079bdcf9761953a097d8fbd3a083b8380fc3214983a814e06bf9834dc/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/4261ee970335d17e6d1c1cbc8b6d2402a666c16644dae7f6782f24e3d7d68743/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/15f16943c5bf0ec9afd2945c260e24f7de1ff555576fbd367c4f8dec04976670/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/03a775bada72d68ce340134e6a9c66143eb30d162fb2a2c9b76517311b3fae95/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/344e1f7586f2a859ac94fe837501c3998b77db74d08413952aadf7185d7307f1/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/9a985bfaf60a777b22873562e244a69264e405a5065e506945822c8e3f9042c3/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/f97e06c5d4bc0f1b91493b22997bc7190f01d8a662258c6caabfbc0fe1710141/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/17cea1c2852eb86604b1ecf17583035b5fafbf7710cec475dd2375cd614440ff/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/96e47856693ea757965a128f5d025b1d49d6a3a44e7d0516a7c3b64f533c9592/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/c23e9fdb13cc80aa5bf1840037b01788a06ba16da86db24a9994d138686dd333/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/1a28ee5b90d0c29d02109efa07e08731eb0d271036aaa6ea272a78d8cc7be75f/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/389a67074df3e3236a452ba712651a6ec45ac28f39b55e72d471f00e25489673/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/af62d4d1a12ba799bf2d8c19d58aab83c14f43f7cbb0464ef11dae9d197d1115/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/6459f918d43af49bf45f2c884d56c86e7b34439497b5799766ff3dbdd9e1f5e6/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/edd0cd8ace0196bd4ad318cec73d43ae626321931b1514da13b29f376c1de5a5/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/cf30510e8ba739639be7a5c6f502019e275f2920d84bfe6647c0394b554c4e8e/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/836a6dab968ac8b2b7e66e9ad36ead8da2a2396670c0c9557345fd978928c58d/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/8ea2948027132d87d273490fbbc3a0bd5ad2fef0b90c43ca589250be03548b17/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/b332797bec9d2ce882b1fe81642753fe89718f5c5832aef8036e2028e2ad18a4/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/408eba947a725ce48510ab598bf4e5c875d931a4a52784dc377ff84dfde1d59e/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/d134abf7cb01e8a08ad44f6cebaef108e58dd8352d7b9d1286d610a1e6dd07a0/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/b79f46ef038a3be39e9491733bb347653a1e80c8160f2c448429fa0cd03af9d4/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/dd63e50552df1d8951375e8dd3daf5ebbd12fd53a8f8070f43317d8693c1c1ea/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/bc5e4c6e46fe40d5bed14343fd6186d4f8bccc9594a2d4d0073d8ad739a2289b/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-220:{mountpoint:/var/lib/containers/storage/overlay/60aceb108efa591db9cec4eacdf4c5ea6578457ad6d1ba9249007302a2f164e9/merged major:0 minor:220 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/e8017fe330cd06cb73724e850111df014e8c307a8b7e84aadd92c0e5c5841803/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/fc457dab91c14ec89ebdf8ef2df941cae0384e9be3ab90564790fe8d0206ecd7/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-240:{mountpoint:/var/lib/containers/storage/overlay/c6b37c53ef39216a936ce439c33d413250d04e4a0ad60045bcd1de3deb0ab659/merged major:0 minor:240 fsType:overlay blockSize:0} overlay_0-242:{mountpoint:/var/lib/containers/storage/overlay/a7c6d9a5f938525b6bfd0f0f4bc4da7ee4d27b51e6344c4e271103584344c77c/merged major:0 minor:242 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/5ec6611996f38499b7eb4aa2b1ebcc4c2737e5483346c5e46899e2b43e7c68e8/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/d26aa6df4ff2733732d976c2a0b00ba73e20ae6e6908ff9321c1bba429b30234/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/0fd012ddaf058645e086f6daaac648d442705aa358227c24e291c0f70bbc62c7/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/b41358223357b4013b8eaa9bbf39bf2bb08b64a7518489a03d7759409996f442/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/2177dc9ef5cb848aa7fd44d1e75f411e662499c6ee8bf570981281d4d0ed437d/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/65cb1823ec493de28fd474a3baf9e086bed495e2b66dead89a0c1cffcef39beb/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/03eab7cbc2e235e7382f351daf7cdfddac3d79a6afdaa942cf5c816be44e3cb6/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-288:{mountpoint:/var/lib/containers/storage/overlay/a1b37c0134eb50eca66fc11336005b7a8440b0be810ad7ff11d5ff4cd95c3f3f/merged major:0 minor:288 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/40f36e11b82a1639d8ec974ffafd4cb8581b417bf075cab52c728725e9118003/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/fc93d02e2a704d4d2b0b99a7a6d6611f16d57d70b05ff4776e761fb4a1150a08/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/59d7068669f71ca8763ba4880116d7b95bd49425da1934c14e99a802471fc972/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/491412d8bbd55e47f2ba6c2bec3673a9e99725abfcbefaee3e1a29c749997470/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/8efb48accd985d68bdb12817958ce29e82209fd29991e95d1cd563df08536779/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/b83f7700e7b64aa3923af7b622c55403d6e0edf8dbfead8bc35001c4350a2b3b/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-331:{mountpoint:/var/lib/containers/storage/overlay/2064c4e7494d7898006ade0ae608fc53cf6f623975aa5a262db5500bde9f6641/merged major:0 minor:331 fsType:overlay blockSize:0} overlay_0-337:{mountpoint:/var/lib/containers/storage/overlay/9f8ce341d06af3525f218f2fed30e7633e63b7c9402e5515c7bd948458e9f679/merged major:0 minor:337 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/849f41909a396ea22d509c7a330470777c0b94aab4f3aa89a9ba2dd960931b67/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-353:{mountpoint:/var/lib/containers/storage/overlay/26c6cd284086632e8836dc15fa9175e1f21aec8bbd0b293c558cc2f66b1ba22f/merged major:0 minor:353 fsType:overlay blockSize:0} overlay_0-355:{mountpoint:/var/lib/containers/storage/overlay/dfd0a362c88a15515148d2c91f8586068566ad4f515e3180ed91644ff3ececb7/merged major:0 minor:355 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/7fd4e9fbce7eb919dcf778c468a3d08105fb5799b2642c8412b26c9bae963c6c/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/416ce571fbfef3b7b7ea9c461bcd3b7ae64b0e16edc92ae6b7f182332b3eab8f/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/fa02b0628dc8075aa22c849298652a38c274659b8c1ef8481755a1f3a87f27c1/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/4eac3911f006e0da3ce49101e09be8d9878263bf8ef9fa13711efc571a6e8b74/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/c3aa290b46f003ddd78099e1fdf9f9c26fb79dd99d8e0ce06414ec6cf95dee93/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/63deebb316e5c27595e0eb1c534d88f59e84b37ed2f0bfe29242122d045f5e0b/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/1a4aa5a08fcaba727afa1bf2f78d272c6bd29a6941428947e48baa0475b852db/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/52174b306d16ee7e099d4754e77b5aa564d644e186af3be737c540db9a04f8c8/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/4eae12d8bcd8f0da02acd92b9b177da0ef89b0da7ccffd728a15935a1c1eb8cb/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/8955e45179aa815f22756e54eb662e9ce3ec1afb009e4e4ed2874c6ce0f7775f/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 18:46:47.477026 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.476425    5232 manager.go:217] Machine: {Timestamp:2026-10-30 18:46:47.475615873 +0000 UTC m=+0.060098863 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098992 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d951ae95e3b218cc4c62743634040 SystemUUID:ec2d951a-e95e-3b21-8cc4-c62743634040 BootID:30cd5602-b9c7-4471-8618-6bd3c678ce46 Filesystems:[{Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46605ee7-f737-431c-8a64-0b430a550d34/volumes/kubernetes.io~projected/kube-api-access-j58xm DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f38a29303ef07383404c0b8ac3f391b516d2c777e07d3b70bc8377f87ead4879/userdata/shm DeviceMajor:0 DeviceMinor:302 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/96a93e14-0292-45a9-8e4b-b5a130ed3d55/volumes/kubernetes.io~projected/kube-api-access-2gmll DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c1f497db70bfc0bd39ee6ff3870b5fdc586c37138c25460d59bc5cc5ae1cdd0c/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/13af2e85-a46f-496a-9151-5c6d649e6fe5/volumes/kubernetes.io~projected/kube-api-access-jjq6z DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-337 DeviceMajor:0 DeviceMinor:337 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b4075d5d-2d2d-4e0e-b913-12715719e711/volumes/kubernetes.io~projected/kube-api-access-kppzz DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:315 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:318 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8f30f6336b2dbfa9f8ab1c3ebbf735e06d291b8723261dd9aea1515c83f7c308/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a768b8cc-62a2-4588-82df-edd68a625830/volumes/kubernetes.io~projected/kube-api-access-nqn7d DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:317 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~projected/kube-api-access-gdv6f DeviceMajor:0 DeviceMinor:283 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9398e679b4eaef3f2b9f8d16b4395486a09a9ecc85e2e2c21edb803c0cde8ad5/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:299 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:313 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:323 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~projected/kube-api-access-wzb8k DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6b153d5855fe6abf809b51f069addb0885ed65fca1333b0df2f85fccea4f461a/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:282 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d22062d6-b0ae-4eed-acd4-f1142e650f41/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:277 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0dce46b5-9bcf-4b26-8126-cac8319d7a9a/volumes/kubernetes.io~projected/kube-api-access-8m4sr DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b8c05cd7abffae36b8d4f9dbfc12e164f19673eb786383f1486e38c2f25feb0b/userdata/shm DeviceMajor:0 DeviceMinor:349 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b202127e-afab-4cac-bee8-e93533bc25d5/volumes/kubernetes.io~projected/kube-api-access-n4b4m DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bbeaac75-73d1-4d21-ad63-7215bb75e8e6/volumes/kubernetes.io~projected/kube-api-access-zwnlq DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~projected/kube-api-access-xbhv5 DeviceMajor:0 DeviceMinor:237 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-288 DeviceMajor:0 DeviceMinor:288 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0267d1961df22fac98e440ea5167ed61574d1fb617f5817dd360955ee2202b74/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:321 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:342 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~projected/kube-api-access-d5nrq DeviceMajor:0 DeviceMinor:348 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eda69974865db84652f4b9fe0fb6d591f9453fb23888e4d332d3a7b7bce95944/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-331 DeviceMajor:0 DeviceMinor:331 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:322 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/13af2e85-a46f-496a-9151-5c6d649e6fe5/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-240 DeviceMajor:0 DeviceMinor:240 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/89987b8e-5b3f-4d20-b82e-643d32d7c004/volumes/kubernetes.io~projected/kube-api-access-74qsz DeviceMajor:0 DeviceMinor:300 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/737ccf945120bedf59271c77ac49cd073328e7ed96c155244050bb4edca5d3a9/userdata/shm DeviceMajor:0 DeviceMinor:325 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:316 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/224cd297-364e-466b-a43e-3ac6a2de7d29/volumes/kubernetes.io~projected/kube-api-access-bhcnv DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/09b151b0942db019c719857b6ffef310a633086971628d385da08d8345221265/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:347 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-353 DeviceMajor:0 DeviceMinor:353 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c12213cefbad597648aca4a0b3a428db72476e55a88d8df3bcc206128326cbc4/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb81cc9b032d077ea698198cdb7a5af6674499023cce89e4cecdddc1341b2886/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-242 DeviceMajor:0 DeviceMinor:242 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:233 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ad9dd4899db49aa05f841cca6ffc25e45b50ddad0da464a1e4d292ecebb6ae86/userdata/shm DeviceMajor:0 DeviceMinor:284 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/224cd297-364e-466b-a43e-3ac6a2de7d29/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:295 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/10b4b395a3d09bf926d67e18f6550ba2d2b9e644522aabb9b8dc30e227ebce9f/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-220 DeviceMajor:0 DeviceMinor:220 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:320 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:318 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b3c313c3-122e-43ce-9d68-106881e7b7a5/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:314 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-355 DeviceMajor:0 DeviceMinor:355 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d417883d4c4541976655c44ca1b09639ea9e465b723673cd5b7da7596a6c76d5/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/20572ffb-cc67-4066-b50c-22489eeed6e7/volumes/kubernetes.io~projected/kube-api-access-fq9pz DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8cb7324d074e007fe1f25f559bfd510fdf3c214b75716ffdc895dc48bac439c5/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f023a1e938200e9c98d0cd0ee31a8a1aa178f625344ece225904f6b0b0d388dd/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3d435197f73cfcab16434af8c55a2916113b4e9d56ad0765f74b36f768649ee8/userdata/shm DeviceMajor:0 DeviceMinor:304 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a224647afab66b3dd230cd4bb5465c8f63d454a6b509c792c0a7fa0dcde13cac/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b/volumes/kubernetes.io~projected/kube-api-access-vh6dp DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46605ee7-f737-431c-8a64-0b430a550d34/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/31c8f647c5cb783a4b9dd0c2165bd62c920ac9da68581886221c4945dd3a2f1f/userdata/shm DeviceMajor:0 DeviceMinor:301 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:319 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a768b8cc-62a2-4588-82df-edd68a625830/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes/kubernetes.io~projected/kube-api-access-rxpvg DeviceMajor:0 DeviceMinor:324 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/20a81c77-50d1-4e5b-a59d-43d455fb842d/volumes/kubernetes.io~projected/kube-api-access-m82st DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a92d5376dba793ee1a873662d6b3cfe8968b2fd4ad2177a12f54e3a4587332a5/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:09b151b0942db01 MacAddress:12:a4:e4:b8:c1:36 Speed:10000 Mtu:8901} {Name:10b4b395a3d09bf MacAddress:76:25:43:a1:9f:69 Speed:10000 Mtu:8901} {Name:31c8f647c5cb783 MacAddress:32:03:81:0f:53:7c Speed:10000 Mtu:8901} {Name:3d435197f73cfca MacAddress:b6:e4:d9:90:b3:94 Speed:10000 Mtu:8901} {Name:6b153d5855fe6ab MacAddress:ce:08:3c:4b:bd:5b Speed:10000 Mtu:8901} {Name:737ccf945120bed MacAddress:ee:de:19:aa:67:66 Speed:10000 Mtu:8901} {Name:a224647afab66b3 MacAddress:0e:8c:00:af:48:db Speed:10000 Mtu:8901} {Name:b8c05cd7abffae3 MacAddress:b6:35:98:d9:84:fc Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:e6:8a:8d:c3 Speed:0 Mtu:9001} {Name:br-int MacAddress:26:8e:1a:d4:e9:95 Speed:0 Mtu:8901} {Name:ens5 MacAddress:06:ff:e6:8a:8d:c3 Speed:0 Mtu:9001} {Name:f023a1e938200e9 MacAddress:d6:03:44:d0:6d:c1 Speed:10000 Mtu:8901} {Name:f38a29303ef0738 MacAddress:6e:27:80:1f:dc:d5 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:1a:ef:e0:f0:a4:d8 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:82:e2:0b:e1:36:b0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:46:47.477206 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477033    5232 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:46:47.477206 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477092    5232 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:46:47.477413 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477386    5232 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:46:47.477534 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477414    5232 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-248.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:46:47.477574 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477535    5232 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:46:47.477574 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477544    5232 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:46:47.477574 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477555    5232 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:46:47.477574 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477574    5232 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:46:47.477680 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477641    5232 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:46:47.477799 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477789    5232 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:46:47.477923 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477914    5232 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:46:47.477957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477949    5232 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:46:47.477990 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477961    5232 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:46:47.477990 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477970    5232 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:46:47.477990 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477979    5232 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:46:47.477990 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.477988    5232 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:46:47.478633 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.478618    5232 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:46:47.478665 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.478635    5232 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:46:47.479110 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.479090    5232 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:46:47.487771 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.487756    5232 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:46:47.487771 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.487768    5232 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:46:47.487901 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.487890    5232 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 18:46:47.488083 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488072    5232 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:46:47.488113 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488101    5232 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:46:47.488213 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488204    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488220    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488234    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488244    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488253    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488262    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488269    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488274    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488280    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:46:47.488290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488287    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:46:47.488520 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488302    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:46:47.488520 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488311    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:46:47.488520 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488336    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:46:47.488520 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.488344    5232 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:46:47.491712 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.491697    5232 server.go:1263] "Started kubelet"
Oct 30 18:46:47.492220 ip-10-0-143-248 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:46:47.494348 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.491969    5232 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:46:47.494348 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.493501    5232 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:46:47.494348 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.493678    5232 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:46:47.494348 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.493903    5232 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:46:47.494348 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.494262    5232 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:46:47.497183 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.497166    5232 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:46:47.497330 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.497239    5232 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:46:47.497514 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.497319    5232 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:46:47.497514 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.497397    5232 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:46:47.497514 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.497408    5232 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:46:47.497514 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.497506    5232 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:46:47.499114 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.497950    5232 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:46:47.499114 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.497965    5232 factory.go:55] Registering systemd factory
Oct 30 18:46:47.499114 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.497975    5232 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:46:47.499114 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.498029    5232 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:40:36 +0000 UTC" deadline="2028-08-17 14:44:32.999768686 +0000 UTC"
Oct 30 18:46:47.499114 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.498053    5232 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15763h57m45.501718516s"
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499646    5232 factory.go:153] Registering CRI-O factory
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499665    5232 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499648    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89987b8e-5b3f-4d20-b82e-643d32d7c004" volumeName="kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499684    5232 factory.go:103] Registering Raw factory
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499685    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a768b8cc-62a2-4588-82df-edd68a625830" volumeName="kubernetes.io/projected/a768b8cc-62a2-4588-82df-edd68a625830-kube-api-access-nqn7d" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499700    5232 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499699    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13af2e85-a46f-496a-9151-5c6d649e6fe5" volumeName="kubernetes.io/projected/13af2e85-a46f-496a-9151-5c6d649e6fe5-kube-api-access-jjq6z" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499714    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20572ffb-cc67-4066-b50c-22489eeed6e7" volumeName="kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-config" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499756    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20572ffb-cc67-4066-b50c-22489eeed6e7" volumeName="kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499771    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499803    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="731eee3b-34bd-4188-9c74-a6a684acce6b" volumeName="kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499840    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499854    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20a81c77-50d1-4e5b-a59d-43d455fb842d" volumeName="kubernetes.io/projected/20a81c77-50d1-4e5b-a59d-43d455fb842d-kube-api-access-m82st" seLinuxMountContext=""
Oct 30 18:46:47.499861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499867    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="731eee3b-34bd-4188-9c74-a6a684acce6b" volumeName="kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499886    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b202127e-afab-4cac-bee8-e93533bc25d5" volumeName="kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-multus-daemon-config" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499898    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd" volumeName="kubernetes.io/secret/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-agent-certs" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499908    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a768b8cc-62a2-4588-82df-edd68a625830" volumeName="kubernetes.io/configmap/a768b8cc-62a2-4588-82df-edd68a625830-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499918    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dce46b5-9bcf-4b26-8126-cac8319d7a9a" volumeName="kubernetes.io/projected/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-kube-api-access-8m4sr" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499932    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499942    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499955    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499967    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1853522-ac6d-49a5-b86b-d8056aadc5d5" volumeName="kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499978    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1853522-ac6d-49a5-b86b-d8056aadc5d5" volumeName="kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.499989    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3c313c3-122e-43ce-9d68-106881e7b7a5" volumeName="kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500000    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13af2e85-a46f-496a-9151-5c6d649e6fe5" volumeName="kubernetes.io/empty-dir/13af2e85-a46f-496a-9151-5c6d649e6fe5-tmp-dir" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500011    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="731eee3b-34bd-4188-9c74-a6a684acce6b" volumeName="kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500023    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b" volumeName="kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-tuned" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500035    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1853522-ac6d-49a5-b86b-d8056aadc5d5" volumeName="kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500047    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22062d6-b0ae-4eed-acd4-f1142e650f41" volumeName="kubernetes.io/empty-dir/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-textfile" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500059    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46605ee7-f737-431c-8a64-0b430a550d34" volumeName="kubernetes.io/projected/46605ee7-f737-431c-8a64-0b430a550d34-kube-api-access-j58xm" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500093    5232 manager.go:319] Starting recovery of all containers
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500098    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500114    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500132    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1853522-ac6d-49a5-b86b-d8056aadc5d5" volumeName="kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500144    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3c313c3-122e-43ce-9d68-106881e7b7a5" volumeName="kubernetes.io/projected/b3c313c3-122e-43ce-9d68-106881e7b7a5-kube-api-access-xbhv5" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500157    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500169    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500182    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500194    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1853522-ac6d-49a5-b86b-d8056aadc5d5" volumeName="kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500208    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20a81c77-50d1-4e5b-a59d-43d455fb842d" volumeName="kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500220    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500231    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500242    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a768b8cc-62a2-4588-82df-edd68a625830" volumeName="kubernetes.io/empty-dir/a768b8cc-62a2-4588-82df-edd68a625830-data-volume" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500256    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1853522-ac6d-49a5-b86b-d8056aadc5d5" volumeName="kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500267    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3c313c3-122e-43ce-9d68-106881e7b7a5" volumeName="kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500280    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22062d6-b0ae-4eed-acd4-f1142e650f41" volumeName="kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500292    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22062d6-b0ae-4eed-acd4-f1142e650f41" volumeName="kubernetes.io/projected/d22062d6-b0ae-4eed-acd4-f1142e650f41-kube-api-access-gdv6f" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500305    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20572ffb-cc67-4066-b50c-22489eeed6e7" volumeName="kubernetes.io/projected/20572ffb-cc67-4066-b50c-22489eeed6e7-kube-api-access-fq9pz" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500316    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500327    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6e3b2c8e-e1e8-499f-9c8b-164e47a57f54" volumeName="kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500339    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="731eee3b-34bd-4188-9c74-a6a684acce6b" volumeName="kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500352    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="731eee3b-34bd-4188-9c74-a6a684acce6b" volumeName="kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500381    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22062d6-b0ae-4eed-acd4-f1142e650f41" volumeName="kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500394    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b" volumeName="kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-tmp" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500407    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="224cd297-364e-466b-a43e-3ac6a2de7d29" volumeName="kubernetes.io/projected/224cd297-364e-466b-a43e-3ac6a2de7d29-kube-api-access-bhcnv" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500422    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="224cd297-364e-466b-a43e-3ac6a2de7d29" volumeName="kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500434    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500456    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="731eee3b-34bd-4188-9c74-a6a684acce6b" volumeName="kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500469    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a768b8cc-62a2-4588-82df-edd68a625830" volumeName="kubernetes.io/secret/a768b8cc-62a2-4588-82df-edd68a625830-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500481    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4075d5d-2d2d-4e0e-b913-12715719e711" volumeName="kubernetes.io/configmap/b4075d5d-2d2d-4e0e-b913-12715719e711-serviceca" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500496    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20572ffb-cc67-4066-b50c-22489eeed6e7" volumeName="kubernetes.io/secret/20572ffb-cc67-4066-b50c-22489eeed6e7-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500517    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500528    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="731eee3b-34bd-4188-9c74-a6a684acce6b" volumeName="kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500541    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4075d5d-2d2d-4e0e-b913-12715719e711" volumeName="kubernetes.io/projected/b4075d5d-2d2d-4e0e-b913-12715719e711-kube-api-access-kppzz" seLinuxMountContext=""
Oct 30 18:46:47.500531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500552    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbeaac75-73d1-4d21-ad63-7215bb75e8e6" volumeName="kubernetes.io/projected/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-kube-api-access-zwnlq" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500563    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500574    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="731eee3b-34bd-4188-9c74-a6a684acce6b" volumeName="kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500586    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1853522-ac6d-49a5-b86b-d8056aadc5d5" volumeName="kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500597    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20a81c77-50d1-4e5b-a59d-43d455fb842d" volumeName="kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500612    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13af2e85-a46f-496a-9151-5c6d649e6fe5" volumeName="kubernetes.io/configmap/13af2e85-a46f-496a-9151-5c6d649e6fe5-config-volume" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500625    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20572ffb-cc67-4066-b50c-22489eeed6e7" volumeName="kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-env-overrides" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500638    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b" volumeName="kubernetes.io/projected/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-kube-api-access-vh6dp" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500649    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="96a93e14-0292-45a9-8e4b-b5a130ed3d55" volumeName="kubernetes.io/projected/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kube-api-access-2gmll" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500660    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3c313c3-122e-43ce-9d68-106881e7b7a5" volumeName="kubernetes.io/configmap/b3c313c3-122e-43ce-9d68-106881e7b7a5-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500672    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd" volumeName="kubernetes.io/configmap/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-konnectivity-ca" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500685    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22062d6-b0ae-4eed-acd4-f1142e650f41" volumeName="kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500697    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13af2e85-a46f-496a-9151-5c6d649e6fe5" volumeName="kubernetes.io/secret/13af2e85-a46f-496a-9151-5c6d649e6fe5-metrics-tls" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500708    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20a81c77-50d1-4e5b-a59d-43d455fb842d" volumeName="kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500721    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46605ee7-f737-431c-8a64-0b430a550d34" volumeName="kubernetes.io/secret/46605ee7-f737-431c-8a64-0b430a550d34-cert" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500733    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b202127e-afab-4cac-bee8-e93533bc25d5" volumeName="kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500744    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dce46b5-9bcf-4b26-8126-cac8319d7a9a" volumeName="kubernetes.io/configmap/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-iptables-alerter-script" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500756    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500777    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500790    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b202127e-afab-4cac-bee8-e93533bc25d5" volumeName="kubernetes.io/projected/b202127e-afab-4cac-bee8-e93533bc25d5-kube-api-access-n4b4m" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500804    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbeaac75-73d1-4d21-ad63-7215bb75e8e6" volumeName="kubernetes.io/empty-dir/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-tmp-dir" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500833    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500845    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a4ac65f-7a27-48a5-9bce-1fe44944d664" volumeName="kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500857    5232 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d22062d6-b0ae-4eed-acd4-f1142e650f41" volumeName="kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-tls" seLinuxMountContext=""
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500868    5232 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:46:47.502722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.500875    5232 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:46:47.530868 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.530824    5232 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:46:47.532127 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.532105    5232 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:46:47.532240 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.532146    5232 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:46:47.532240 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.532167    5232 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:46:47.532240 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.532223    5232 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 18:46:47.540450 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.540419    5232 generic.go:356] "Generic (PLEG): container finished" podID="d22062d6-b0ae-4eed-acd4-f1142e650f41" containerID="b24ee5d07e1133eba403c462e1dc64108e1845bca85be96175e5ee37e5db9e8c" exitCode=0
Oct 30 18:46:47.547674 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.547647    5232 generic.go:356] "Generic (PLEG): container finished" podID="58894da434aca988e2149b0960c93e80" containerID="244a61cf726bb4b7443b3894878eab0faacf28a0c1f0f3359945ce70f5bed6b6" exitCode=0
Oct 30 18:46:47.558043 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.558020    5232 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="3ab96e0fac7e10ce6a271f9b974090048dc08bed049a0de10ca812600d27acb3" exitCode=0
Oct 30 18:46:47.558043 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.558041    5232 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="365049c1f711bff37e6886c83f4b9d1acf0767f893e068687b8c81a7c312e136" exitCode=0
Oct 30 18:46:47.558043 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.558047    5232 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="ef188b1fdc7a347c84532ced574f6f88bffff31ebc3e28caeddbd21734d5f477" exitCode=0
Oct 30 18:46:47.558218 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.558055    5232 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="9545194544c4c1e44e4b71177f094d0ab364432deb1a7ed442c59e9331f9e2df" exitCode=0
Oct 30 18:46:47.558218 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.558074    5232 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="9c45be752cc8b07f1f1a9434ea858d58e4eedd7cd5842f8e3c275998fd80de54" exitCode=0
Oct 30 18:46:47.558218 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.558079    5232 generic.go:356] "Generic (PLEG): container finished" podID="20a81c77-50d1-4e5b-a59d-43d455fb842d" containerID="a34ce64c298b4597d0f9c7ed8f78e0a66807859d8b4d6f66aae657a560a84eeb" exitCode=0
Oct 30 18:46:47.563344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.563316    5232 generic.go:356] "Generic (PLEG): container finished" podID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038" exitCode=0
Oct 30 18:46:47.589300 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.589277    5232 manager.go:324] Recovery completed
Oct 30 18:46:47.597452 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.597420    5232 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:46:47.606680 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.606666    5232 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:46:47.608757 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.608740    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:46:47.608849 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.608770    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:46:47.608849 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.608781    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:46:47.611342 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611295    5232 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:46:47.611342 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611309    5232 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:46:47.611342 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611312    5232 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:46:47.611342 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611324    5232 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:46:47.611528 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611433    5232 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 18:46:47.611528 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611443    5232 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 18:46:47.611528 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611459    5232 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 18:46:47.611528 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611465    5232 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 18:46:47.611528 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611471    5232 policy_none.go:50] "Start"
Oct 30 18:46:47.611528 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611483    5232 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:46:47.611528 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.611492    5232 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:46:47.612375 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.612361    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:46:47.612437 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.612387    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:46:47.612437 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.612399    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:46:47.613460 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.613445    5232 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 18:46:47.613495 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.613463    5232 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 18:46:47.613495 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.613469    5232 policy_none.go:44] "Start"
Oct 30 18:46:47.617594 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.617572    5232 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:46:47.620141 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620130    5232 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:46:47.620182 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.620154    5232 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:46:47.620182 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620163    5232 server.go:87] "Starting device plugin registration server"
Oct 30 18:46:47.620423 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620412    5232 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:46:47.620459 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620426    5232 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:46:47.620536 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620514    5232 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:46:47.620648 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620621    5232 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:46:47.620710 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620657    5232 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:46:47.620710 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620664    5232 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:46:47.620805 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.620719    5232 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:46:47.620646071Z","UUID":"d9ba5f3b-6a3f-4f48-a312-636b8df656e2","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:46:47.622486 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.622467    5232 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:46:47.622567 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.622519    5232 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:46:47.623533 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.623503    5232 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:46:47.623612 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.623555    5232 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-248.us-west-2.compute.internal\" not found"
Oct 30 18:46:47.632287 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.632253    5232 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"]
Oct 30 18:46:47.632384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.632373    5232 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:46:47.633399 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.633382    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:46:47.633461 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.633410    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:46:47.633461 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.633422    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:46:47.633695 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.633684    5232 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:46:47.633850 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.633837    5232 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:46:47.634643 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.634626    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:46:47.634643 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.634635    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:46:47.634764 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.634648    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:46:47.634764 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.634658    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:46:47.634764 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.634662    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:46:47.634764 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.634672    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:46:47.634970 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.634798    5232 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.635069 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.635039    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"58894da434aca988e2149b0960c93e80","Type":"ContainerStarted","Data":"f6ce8d1861ca09ab9ad591a308d68fc925ae2872caaee6b4131907c4f1e25c2a"}
Oct 30 18:46:47.635130 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.635081    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"58894da434aca988e2149b0960c93e80","Type":"ContainerDied","Data":"244a61cf726bb4b7443b3894878eab0faacf28a0c1f0f3359945ce70f5bed6b6"}
Oct 30 18:46:47.635130 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.635104    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"58894da434aca988e2149b0960c93e80","Type":"ContainerStarted","Data":"d417883d4c4541976655c44ca1b09639ea9e465b723673cd5b7da7596a6c76d5"}
Oct 30 18:46:47.635130 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.635121    5232 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:46:47.635130 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.635123    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"d0bf5677d5ce41c447fb39800129b147","Type":"ContainerStarted","Data":"bc23fcbd9fd02337e3d3aee7f797f4acaae7754c0b3e06cc57621a0c43a1b162"}
Oct 30 18:46:47.635305 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.635141    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal" event={"ID":"d0bf5677d5ce41c447fb39800129b147","Type":"ContainerStarted","Data":"fb81cc9b032d077ea698198cdb7a5af6674499023cce89e4cecdddc1341b2886"}
Oct 30 18:46:47.636023 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.636005    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:46:47.636109 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.636028    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:46:47.636109 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.636038    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:46:47.636172 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.636150    5232 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-248.us-west-2.compute.internal\" not found" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.682882 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.682860    5232 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 18:46:47.684870 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.684853    5232 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:46:47.684940 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.684880    5232 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:46:47.720682 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.720659    5232 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:46:47.724352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.724336    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:46:47.724415 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.724364    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:46:47.724415 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.724375    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:46:47.724415 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.724414    5232 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.734087 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.734069    5232 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:46:47.778154 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.778126    5232 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:46:47.783107 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.783085    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bf5677d5ce41c447fb39800129b147-config\") pod \"kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"d0bf5677d5ce41c447fb39800129b147\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.783185 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.783119    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.783185 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.783146    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.798373 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.798350    5232 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.833443 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.833421    5232 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.833500 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.833491    5232 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.834054 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.834037    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeNotReady"
Oct 30 18:46:47.834116 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.834071    5232 setters.go:546] "Node became not ready" node="ip-10-0-143-248.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-30T18:46:47Z","lastTransitionTime":"2026-10-30T18:46:47Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 30 18:46:47.864735 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.864667    5232 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:46:47.864844 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.864744    5232 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.864844 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.864764    5232 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.883600 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.883563    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bf5677d5ce41c447fb39800129b147-config\") pod \"kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"d0bf5677d5ce41c447fb39800129b147\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.883666 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.883607    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.883666 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.883625    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.883763 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.883667    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.883763 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.883669    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bf5677d5ce41c447fb39800129b147-config\") pod \"kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"d0bf5677d5ce41c447fb39800129b147\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.883763 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.883687    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58894da434aca988e2149b0960c93e80-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" (UID: \"58894da434aca988e2149b0960c93e80\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:47.890157 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.890136    5232 nodeinfomanager.go:471] existing CSINode "ip-10-0-143-248.us-west-2.compute.internal" is owned by different node (oldNodeID="86d890c6-bfbc-4312-9508-8309055992d6", newNodeID=""), cleaning up...
Oct 30 18:46:47.956154 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:47.956127    5232 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:46:47.956292 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:47.956174    5232 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:48.108232 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.108204    5232 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-248.us-west-2.compute.internal" event="NodeReady"
Oct 30 18:46:48.108394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.108301    5232 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:46:48.478879 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.478848    5232 apiserver.go:52] "Watching apiserver"
Oct 30 18:46:48.583215 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.583188    5232 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:48.583397 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.583334    5232 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:48.625657 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.625630    5232 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:46:48.627399 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.627361    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cn7zd","openshift-dns/node-resolver-6rdvd","openshift-network-diagnostics/network-check-target-f8whs","openshift-console/console-7b9db974cb-slw8w","kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal","openshift-multus/network-metrics-daemon-29t9c","openshift-cluster-node-tuning-operator/tuned-t75nr","openshift-console/console-5f888ff8d9-78xgg","openshift-network-operator/iptables-alerter-sg2mz","openshift-insights/insights-runtime-extractor-zhb5w","openshift-multus/multus-additional-cni-plugins-ns79r","openshift-image-registry/node-ca-phfz4","kube-system/konnectivity-agent-xhv7g","openshift-monitoring/node-exporter-8jv7v","openshift-multus/multus-9jtw5","openshift-ovn-kubernetes/ovnkube-node-dchr9","kube-system/global-pull-secret-syncer-kml2f","openshift-ingress-canary/ingress-canary-tkr8v","openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal","openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:46:48.643088 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.643069    5232 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:46:48.643193 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:48.643110    5232 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:48.643608 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.643589    5232 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:46:48.643641 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:46:48.643628    5232 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-248.us-west-2.compute.internal"
Oct 30 18:46:48.680163 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.680145    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.680498 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.680483    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:46:48.680721 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.680706    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:46:48.681336 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.681319    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-77vpl\""
Oct 30 18:46:48.683000 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.682985    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.683189 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.683173    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:46:48.683250 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.683181    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-29qlm\""
Oct 30 18:46:48.683250 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.683224    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.683315 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.683263    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hz6tx\""
Oct 30 18:46:48.683468 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.683454    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-crxtc\""
Oct 30 18:46:48.683683 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.683669    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:46:48.683780 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.683768    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgxcg\""
Oct 30 18:46:48.685287 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685240    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:46:48.685287 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685246    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.685425 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685352    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.685425 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685353    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.685425 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685381    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.685583 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685481    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:46:48.685583 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685518    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.685583 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685525    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.685583 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685577    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:46:48.685751 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685740    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.685869 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.685858    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.686031 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.686019    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lj6ts\""
Oct 30 18:46:48.686559 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.686547    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:46:48.686754 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.686740    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.686803 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.686747    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:46:48.686803 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.686764    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n4vwp\""
Oct 30 18:46:48.687239 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.687225    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.687369 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.687354    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5tsjg\""
Oct 30 18:46:48.687431 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.687418    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:46:48.687470 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.687453    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.688312 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688297    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:46:48.688456 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688442    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bpjwm\""
Oct 30 18:46:48.688456 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688453    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-r47g8\""
Oct 30 18:46:48.688590 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688576    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.688633 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688580    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.688721 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688707    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-56wtl\""
Oct 30 18:46:48.688759 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688741    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:46:48.688803 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688764    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:46:48.688974 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.688963    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:46:48.699657 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.699642    5232 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:46:48.781543 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781404    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:46:48.781543 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781422    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 18:46:48.781543 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781442    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 18:46:48.781543 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781438    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 18:46:48.781543 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781437    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.781543 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781474    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.781842 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781712    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 18:46:48.781842 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781762    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.781901 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781849    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.781901 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781887    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.781961 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781945    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.782017 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.781994    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:46:48.782172 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.782152    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 18:46:48.782436 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.782418    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:46:48.782526 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.782439    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:46:48.782526 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.782468    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.784693    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.784746    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.784798    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.784830    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.784842    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.784876    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.784933    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.784850    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.785089    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.785120    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.785130    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.785215    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 18:46:48.785385 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.785237    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q9tbn\""
Oct 30 18:46:48.786045 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.785535    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 18:46:48.787281 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787261    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-vvlg2\""
Oct 30 18:46:48.787373 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787267    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 18:46:48.787373 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787295    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cntpx\""
Oct 30 18:46:48.787373 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787318    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 18:46:48.787485 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787399    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 18:46:48.787692 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787675    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-kubelet-config\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:48.787746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787703    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.787746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787719    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.787746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787736    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gmll\" (UniqueName: \"kubernetes.io/projected/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kube-api-access-2gmll\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.787897 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787757    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-system-cni-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.787897 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787827    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.787897 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787855    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787922    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m4sr\" (UniqueName: \"kubernetes.io/projected/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-kube-api-access-8m4sr\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:46:48.788053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787957    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787962    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-hosts-file\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:46:48.788053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787987    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-ovn\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.788053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.787929    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788015    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788043    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m82st\" (UniqueName: \"kubernetes.io/projected/20a81c77-50d1-4e5b-a59d-43d455fb842d-kube-api-access-m82st\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788062    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788078    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j58xm\" (UniqueName: \"kubernetes.io/projected/46605ee7-f737-431c-8a64-0b430a550d34-kube-api-access-j58xm\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788087    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788094    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788147    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-sys\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788175    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-script-lib\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788219    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788250    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788276    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788356    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788377    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-wtmp\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788392    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-netd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788409    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20572ffb-cc67-4066-b50c-22489eeed6e7-ovn-node-metrics-cert\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788436    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788441    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788448    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-script-lib\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788469    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-registration-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788493    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-sys-fs\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788519    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a768b8cc-62a2-4588-82df-edd68a625830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788585    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788590    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-lib-modules\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788621    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh6dp\" (UniqueName: \"kubernetes.io/projected/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-kube-api-access-vh6dp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788655    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788681    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdv6f\" (UniqueName: \"kubernetes.io/projected/d22062d6-b0ae-4eed-acd4-f1142e650f41-kube-api-access-gdv6f\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788629    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20572ffb-cc67-4066-b50c-22489eeed6e7-ovn-node-metrics-cert\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788706    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjq6z\" (UniqueName: \"kubernetes.io/projected/13af2e85-a46f-496a-9151-5c6d649e6fe5-kube-api-access-jjq6z\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788732    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-multus-certs\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788742    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a768b8cc-62a2-4588-82df-edd68a625830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788728    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788758    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.788778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788784    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788839    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788880    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-modprobe-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.788901    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysconfig\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789031    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-tuned\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789049    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789065    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4075d5d-2d2d-4e0e-b913-12715719e711-serviceca\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789079    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789094    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5nrq\" (UniqueName: \"kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789111    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-conf\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789154    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-tuned\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789171    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-original-pull-secret\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789210    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789287    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789306    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789406    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-multus\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789458    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789504    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-run\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789539    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-agent-certs\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789541    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.789630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789569    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a768b8cc-62a2-4588-82df-edd68a625830-data-volume\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789671    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-tmp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789677    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4075d5d-2d2d-4e0e-b913-12715719e711-serviceca\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789708    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-log-socket\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789726    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789737    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-tmp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789730    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a768b8cc-62a2-4588-82df-edd68a625830-data-volume\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789744    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789774    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a768b8cc-62a2-4588-82df-edd68a625830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789790    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-metrics-client-ca\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789806    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-cnibin\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789836    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789847    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-agent-certs\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789867    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3c313c3-122e-43ce-9d68-106881e7b7a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789887    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789898    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789902    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789904    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-etc-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789940    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq9pz\" (UniqueName: \"kubernetes.io/projected/20572ffb-cc67-4066-b50c-22489eeed6e7-kube-api-access-fq9pz\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789956    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4075d5d-2d2d-4e0e-b913-12715719e711-host\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789970    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.789990    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790015    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13af2e85-a46f-496a-9151-5c6d649e6fe5-metrics-tls\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790015    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-metrics-client-ca\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790018    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3c313c3-122e-43ce-9d68-106881e7b7a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790028    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-dbus\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790056    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/224cd297-364e-466b-a43e-3ac6a2de7d29-metrics-certs\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790068    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790069    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790109    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-var-lib-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790133    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-netns\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790158    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-hostroot\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790183    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-systemd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790204    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13af2e85-a46f-496a-9151-5c6d649e6fe5-metrics-tls\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790246    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790308    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzb8k\" (UniqueName: \"kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790345    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.790361 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790370    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-konnectivity-ca\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790396    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a768b8cc-62a2-4588-82df-edd68a625830-crio-socket\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790410    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790447    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-tls\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790464    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-device-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790506    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-kubernetes\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790552    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqn7d\" (UniqueName: \"kubernetes.io/projected/a768b8cc-62a2-4588-82df-edd68a625830-kube-api-access-nqn7d\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790570    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd-konnectivity-ca\") pod \"konnectivity-agent-xhv7g\" (UID: \"cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd\") " pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790587    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-sys\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790652    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790671    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-node-log\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790686    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-var-lib-kubelet\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790700    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-tmp-dir\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790738    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-kubelet\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790796    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-slash\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790847    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kppzz\" (UniqueName: \"kubernetes.io/projected/b4075d5d-2d2d-4e0e-b913-12715719e711-kube-api-access-kppzz\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790864    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790864    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-tmp-dir\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790893    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-accelerators-collector-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790908    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-socket-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790957    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwnlq\" (UniqueName: \"kubernetes.io/projected/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-kube-api-access-zwnlq\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791005    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.790986    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-textfile\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791048    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791081    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcnv\" (UniqueName: \"kubernetes.io/projected/224cd297-364e-466b-a43e-3ac6a2de7d29-kube-api-access-bhcnv\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791091    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-textfile\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791097    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791132    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-cnibin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791158    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-systemd\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791188    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxpvg\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791217    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791270    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791303    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791321    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-k8s-cni-cncf-io\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791340    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791343    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791396    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791430    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-iptables-alerter-script\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791445    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791458    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-conf-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791485    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-host\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791513    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-root\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791566    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791567    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-env-overrides\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791590    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-iptables-alerter-script\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791606    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:48.791595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791615    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791636    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-host-slash\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791657    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791672    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791713    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-env-overrides\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791763    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-etc-kubernetes\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791781    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-socket-dir-parent\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791795    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3c313c3-122e-43ce-9d68-106881e7b7a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791797    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791841    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791932    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.791992    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792008    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792012    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792051    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792073    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792116    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792146    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-bin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792162    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792164    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbhv5\" (UniqueName: \"kubernetes.io/projected/b3c313c3-122e-43ce-9d68-106881e7b7a5-kube-api-access-xbhv5\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792192    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792216    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792247    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4b4m\" (UniqueName: \"kubernetes.io/projected/b202127e-afab-4cac-bee8-e93533bc25d5-kube-api-access-n4b4m\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792268    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-system-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792285    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792302    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-cni-binary-copy\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792385    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792396    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792465    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-cni-binary-copy\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792526    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-netns\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792547    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46605ee7-f737-431c-8a64-0b430a550d34-cert\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792568    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792583    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-bin\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792616    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13af2e85-a46f-496a-9151-5c6d649e6fe5-tmp-dir\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792666    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-multus-daemon-config\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792683    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/13af2e85-a46f-496a-9151-5c6d649e6fe5-config-volume\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792708    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13af2e85-a46f-496a-9151-5c6d649e6fe5-tmp-dir\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792748    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-kubelet\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792785    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-systemd-units\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792795    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46605ee7-f737-431c-8a64-0b430a550d34-cert\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792828    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-config\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792860    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792879    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b202127e-afab-4cac-bee8-e93533bc25d5-multus-daemon-config\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792895    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/13af2e85-a46f-496a-9151-5c6d649e6fe5-config-volume\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.792902    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.793038    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20a81c77-50d1-4e5b-a59d-43d455fb842d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.793044    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20572ffb-cc67-4066-b50c-22489eeed6e7-ovnkube-config\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.793295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.793176    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.829778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.829755    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 18:46:48.829778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.829776    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6zff9\""
Oct 30 18:46:48.829921 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.829754    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7tn3b1m8racr8\""
Oct 30 18:46:48.829921 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.829755    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 18:46:48.829921 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.829796    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 18:46:48.830236 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830219    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:46:48.830310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830235    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:46:48.830310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830221    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lk2tt\""
Oct 30 18:46:48.830310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830262    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjgl9\""
Oct 30 18:46:48.830310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830264    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jrxvz\""
Oct 30 18:46:48.830310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830277    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k86kp\""
Oct 30 18:46:48.830310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830284    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:46:48.830660 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830641    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 18:46:48.830743 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830727    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 18:46:48.830793 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830730    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.830793 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830746    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-tls\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.830974 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.830957    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 18:46:48.832835 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.832806    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.834012 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.833995    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.835516 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.835499    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.839086 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.839048    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.839152 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.839086    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.839152 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.839141    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.839251 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.839194    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.840339 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.840325    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a768b8cc-62a2-4588-82df-edd68a625830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.851158 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.851135    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m82st\" (UniqueName: \"kubernetes.io/projected/20a81c77-50d1-4e5b-a59d-43d455fb842d-kube-api-access-m82st\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.851305 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.851288    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gmll\" (UniqueName: \"kubernetes.io/projected/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kube-api-access-2gmll\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.852184 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.852167    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m4sr\" (UniqueName: \"kubernetes.io/projected/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-kube-api-access-8m4sr\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:46:48.856722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.856706    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j58xm\" (UniqueName: \"kubernetes.io/projected/46605ee7-f737-431c-8a64-0b430a550d34-kube-api-access-j58xm\") pod \"ingress-canary-tkr8v\" (UID: \"46605ee7-f737-431c-8a64-0b430a550d34\") " pod="openshift-ingress-canary/ingress-canary-tkr8v"
Oct 30 18:46:48.858658 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.858644    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74qsz\" (UniqueName: \"kubernetes.io/projected/89987b8e-5b3f-4d20-b82e-643d32d7c004-kube-api-access-74qsz\") pod \"network-check-target-f8whs\" (UID: \"89987b8e-5b3f-4d20-b82e-643d32d7c004\") " pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:48.860876 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.860856    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhcnv\" (UniqueName: \"kubernetes.io/projected/224cd297-364e-466b-a43e-3ac6a2de7d29-kube-api-access-bhcnv\") pod \"network-metrics-daemon-29t9c\" (UID: \"224cd297-364e-466b-a43e-3ac6a2de7d29\") " pod="openshift-multus/network-metrics-daemon-29t9c"
Oct 30 18:46:48.860944 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.860862    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4b4m\" (UniqueName: \"kubernetes.io/projected/b202127e-afab-4cac-bee8-e93533bc25d5-kube-api-access-n4b4m\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.861111 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.861092    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwnlq\" (UniqueName: \"kubernetes.io/projected/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-kube-api-access-zwnlq\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:46:48.861143 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.861110    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kppzz\" (UniqueName: \"kubernetes.io/projected/b4075d5d-2d2d-4e0e-b913-12715719e711-kube-api-access-kppzz\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:46:48.861236 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.861222    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbhv5\" (UniqueName: \"kubernetes.io/projected/b3c313c3-122e-43ce-9d68-106881e7b7a5-kube-api-access-xbhv5\") pod \"prometheus-operator-9cd89b74c-mpn8v\" (UID: \"b3c313c3-122e-43ce-9d68-106881e7b7a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mpn8v"
Oct 30 18:46:48.861491 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.861476    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5nrq\" (UniqueName: \"kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq\") pod \"console-5f888ff8d9-78xgg\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") " pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:48.862111 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.862092    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxpvg\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg\") pod \"prometheus-k8s-0\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:46:48.862111 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.862101    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqn7d\" (UniqueName: \"kubernetes.io/projected/a768b8cc-62a2-4588-82df-edd68a625830-kube-api-access-nqn7d\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.862216 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.862105    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh6dp\" (UniqueName: \"kubernetes.io/projected/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-kube-api-access-vh6dp\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.862216 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.862168    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq9pz\" (UniqueName: \"kubernetes.io/projected/20572ffb-cc67-4066-b50c-22489eeed6e7-kube-api-access-fq9pz\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.862597 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.862583    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdv6f\" (UniqueName: \"kubernetes.io/projected/d22062d6-b0ae-4eed-acd4-f1142e650f41-kube-api-access-gdv6f\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.866169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.866154    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjq6z\" (UniqueName: \"kubernetes.io/projected/13af2e85-a46f-496a-9151-5c6d649e6fe5-kube-api-access-jjq6z\") pod \"dns-default-cn7zd\" (UID: \"13af2e85-a46f-496a-9151-5c6d649e6fe5\") " pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:48.867398 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.867381    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzb8k\" (UniqueName: \"kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k\") pod \"console-7b9db974cb-slw8w\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") " pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:48.893297 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893263    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-multus-certs\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.893423 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893306    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-modprobe-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.893423 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893331    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysconfig\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.893423 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893356    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.893423 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893376    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-conf\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.893423 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893400    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-multus\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.893423 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893373    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-multus-certs\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893425    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysconfig\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893458    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-modprobe-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893465    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893462    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-run\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893525    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-multus\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893539    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-run\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893539    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-conf\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893544    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-log-socket\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893573    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-log-socket\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893589    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-cnibin\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893616    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-etc-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893635    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4075d5d-2d2d-4e0e-b913-12715719e711-host\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893651    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893672    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-cnibin\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893675    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-dbus\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:48.893685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893691    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-etc-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893700    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4075d5d-2d2d-4e0e-b913-12715719e711-host\") pod \"node-ca-phfz4\" (UID: \"b4075d5d-2d2d-4e0e-b913-12715719e711\") " pod="openshift-image-registry/node-ca-phfz4"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893708    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893715    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-var-lib-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893739    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-netns\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893765    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-hostroot\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893774    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-dbus\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893784    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-var-lib-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893787    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-netns\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893786    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-systemd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893827    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-hostroot\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893835    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-systemd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893839    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893863    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a768b8cc-62a2-4588-82df-edd68a625830-crio-socket\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893880    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-device-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893892    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893894    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-kubernetes\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893922    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-sys\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893937    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-kubernetes\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893940    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a768b8cc-62a2-4588-82df-edd68a625830-crio-socket\") pod \"insights-runtime-extractor-zhb5w\" (UID: \"a768b8cc-62a2-4588-82df-edd68a625830\") " pod="openshift-insights/insights-runtime-extractor-zhb5w"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893948    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-device-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893949    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-node-log\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893971    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-sys\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893978    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-node-log\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.893988    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-var-lib-kubelet\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894009    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-kubelet\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894025    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-slash\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894040    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-socket-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894060    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-kubelet\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894062    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-cnibin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894082    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-slash\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894093    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-var-lib-kubelet\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894099    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-systemd\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894110    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-socket-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894124    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-cnibin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894128    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-k8s-cni-cncf-io\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894153    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-systemd\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894159    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-run-k8s-cni-cncf-io\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894159    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894183    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-conf-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894197    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-host\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894220    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-root\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894183    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-openvswitch\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894243    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-conf-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.894227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894246    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-host-slash\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894278    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-host\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894281    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-etc-kubernetes\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894285    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-root\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894298    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-socket-dir-parent\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894315    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894324    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dce46b5-9bcf-4b26-8126-cac8319d7a9a-host-slash\") pod \"iptables-alerter-sg2mz\" (UID: \"0dce46b5-9bcf-4b26-8126-cac8319d7a9a\") " pod="openshift-network-operator/iptables-alerter-sg2mz"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894339    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894360    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-etc-kubernetes\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894368    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-bin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894396    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-system-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894367    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-socket-dir-parent\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894425    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894400    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894364    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894439    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-system-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894450    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-netns\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894481    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-bin\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894507    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-bin\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894400    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-host-var-lib-cni-bin\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894483    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b202127e-afab-4cac-bee8-e93533bc25d5-multus-cni-dir\") pod \"multus-9jtw5\" (UID: \"b202127e-afab-4cac-bee8-e93533bc25d5\") " pod="openshift-multus/multus-9jtw5"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894507    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-kubelet\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894536    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-kubelet\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894541    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-systemd-units\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894483    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-run-netns\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894573    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-systemd-units\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894593    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-kubelet-config\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894612    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-system-cni-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894631    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-hosts-file\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894645    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-ovn\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894668    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894694    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e3b2c8e-e1e8-499f-9c8b-164e47a57f54-kubelet-config\") pod \"global-pull-secret-syncer-kml2f\" (UID: \"6e3b2c8e-e1e8-499f-9c8b-164e47a57f54\") " pod="kube-system/global-pull-secret-syncer-kml2f"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894704    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bbeaac75-73d1-4d21-ad63-7215bb75e8e6-hosts-file\") pod \"node-resolver-6rdvd\" (UID: \"bbeaac75-73d1-4d21-ad63-7215bb75e8e6\") " pod="openshift-dns/node-resolver-6rdvd"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894712    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20a81c77-50d1-4e5b-a59d-43d455fb842d-system-cni-dir\") pod \"multus-additional-cni-plugins-ns79r\" (UID: \"20a81c77-50d1-4e5b-a59d-43d455fb842d\") " pod="openshift-multus/multus-additional-cni-plugins-ns79r"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894722    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-sys\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894705    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-run-ovn\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894743    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-wtmp\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894759    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-netd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894776    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-etc-sysctl-d\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894781    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-registration-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894804    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-sys-fs\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894829    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22062d6-b0ae-4eed-acd4-f1142e650f41-node-exporter-wtmp\") pod \"node-exporter-8jv7v\" (UID: \"d22062d6-b0ae-4eed-acd4-f1142e650f41\") " pod="openshift-monitoring/node-exporter-8jv7v"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894833    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-sys\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894832    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20572ffb-cc67-4066-b50c-22489eeed6e7-host-cni-netd\") pod \"ovnkube-node-dchr9\" (UID: \"20572ffb-cc67-4066-b50c-22489eeed6e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894841    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-registration-dir\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894874    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96a93e14-0292-45a9-8e4b-b5a130ed3d55-sys-fs\") pod \"aws-ebs-csi-driver-node-9v6gd\" (UID: \"96a93e14-0292-45a9-8e4b-b5a130ed3d55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v6gd"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894843    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-lib-modules\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.895610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.894926    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b-lib-modules\") pod \"tuned-t75nr\" (UID: \"7d3f34a6-a58f-44d7-9cf1-8e9a5f8c300b\") " pod="openshift-cluster-node-tuning-operator/tuned-t75nr"
Oct 30 18:46:48.938397 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:48.938378    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:46:49.093035 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:49.093005    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b9db974cb-slw8w"]
Oct 30 18:46:49.590161 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:49.590132    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cn7zd"
Oct 30 18:46:50.897282 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:50.897243    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:46:50.976742 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:50.976708    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:50.982203 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:50.982178    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:52.106838 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:52.106794    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:46:52.600783 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:52.600749    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerStarted","Data":"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"}
Oct 30 18:46:52.600997 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:52.600833    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerStarted","Data":"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"}
Oct 30 18:46:53.156355 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:53.156312    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:46:53.264570 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:53.264541    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xhv7g"
Oct 30 18:46:55.616755 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:55.616719    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerStarted","Data":"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"}
Oct 30 18:46:55.617159 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:55.616784    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerStarted","Data":"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"}
Oct 30 18:46:55.617159 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:55.616832    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerStarted","Data":"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"}
Oct 30 18:46:55.617159 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:55.616864    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerStarted","Data":"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"}
Oct 30 18:46:56.094103 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:56.094078    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:57.213837 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:57.213789    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:57.214296 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:57.214102    5232 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:46:57.621066 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:57.621039    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dchr9"
Oct 30 18:46:57.621905 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:46:57.621882    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:47:03.161881 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:03.161851    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:47:14.606888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.606803    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7b9db974cb-slw8w" podUID="b1853522-ac6d-49a5-b86b-d8056aadc5d5" containerName="console" containerID="cri-o://5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166" gracePeriod=15
Oct 30 18:47:14.852255 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.852224    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:47:14.887648 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.887585    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config\") pod \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") "
Oct 30 18:47:14.887648 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.887630    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp\") pod \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") "
Oct 30 18:47:14.887839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.887651    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca\") pod \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") "
Oct 30 18:47:14.887839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.887671    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config\") pod \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") "
Oct 30 18:47:14.887942 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.887868    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert\") pod \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") "
Oct 30 18:47:14.887990 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.887973    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert\") pod \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") "
Oct 30 18:47:14.888030 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.888016    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k\" (UniqueName: \"kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k\") pod \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\" (UID: \"b1853522-ac6d-49a5-b86b-d8056aadc5d5\") "
Oct 30 18:47:14.888079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.887974    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp" pod "b1853522-ac6d-49a5-b86b-d8056aadc5d5" (UID: "b1853522-ac6d-49a5-b86b-d8056aadc5d5"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:47:14.888120 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.888093    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca" pod "b1853522-ac6d-49a5-b86b-d8056aadc5d5" (UID: "b1853522-ac6d-49a5-b86b-d8056aadc5d5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:14.888200 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.888112    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config" pod "b1853522-ac6d-49a5-b86b-d8056aadc5d5" (UID: "b1853522-ac6d-49a5-b86b-d8056aadc5d5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:14.888294 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.888249    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert" pod "b1853522-ac6d-49a5-b86b-d8056aadc5d5" (UID: "b1853522-ac6d-49a5-b86b-d8056aadc5d5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:14.888338 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.888295    5232 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-config\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:14.888338 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.888315    5232 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1853522-ac6d-49a5-b86b-d8056aadc5d5-tmp\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:14.888338 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.888330    5232 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:14.890026 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.889996    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert" pod "b1853522-ac6d-49a5-b86b-d8056aadc5d5" (UID: "b1853522-ac6d-49a5-b86b-d8056aadc5d5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:14.890137 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.890026    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k" pod "b1853522-ac6d-49a5-b86b-d8056aadc5d5" (UID: "b1853522-ac6d-49a5-b86b-d8056aadc5d5"). InnerVolumeSpecName "kube-api-access-wzb8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:14.890137 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.890038    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config" pod "b1853522-ac6d-49a5-b86b-d8056aadc5d5" (UID: "b1853522-ac6d-49a5-b86b-d8056aadc5d5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:14.989561 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.989522    5232 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-serving-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:14.989561 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.989557    5232 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1853522-ac6d-49a5-b86b-d8056aadc5d5-oauth-serving-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:14.989755 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.989570    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzb8k\" (UniqueName: \"kubernetes.io/projected/b1853522-ac6d-49a5-b86b-d8056aadc5d5-kube-api-access-wzb8k\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:14.989755 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:14.989583    5232 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1853522-ac6d-49a5-b86b-d8056aadc5d5-console-oauth-config\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:15.683066 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.683033    5232 generic.go:356] "Generic (PLEG): container finished" podID="b1853522-ac6d-49a5-b86b-d8056aadc5d5" containerID="5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166" exitCode=0
Oct 30 18:47:15.683558 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.683085    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9db974cb-slw8w"
Oct 30 18:47:15.683558 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.683116    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9db974cb-slw8w" event={"ID":"b1853522-ac6d-49a5-b86b-d8056aadc5d5","Type":"ContainerDied","Data":"5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166"}
Oct 30 18:47:15.683558 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.683165    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9db974cb-slw8w" event={"ID":"b1853522-ac6d-49a5-b86b-d8056aadc5d5","Type":"ContainerDied","Data":"6b153d5855fe6abf809b51f069addb0885ed65fca1333b0df2f85fccea4f461a"}
Oct 30 18:47:15.683558 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.683197    5232 scope.go:122] "RemoveContainer" containerID="5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166"
Oct 30 18:47:15.691001 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.690983    5232 scope.go:122] "RemoveContainer" containerID="5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166"
Oct 30 18:47:15.691301 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:47:15.691279    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166\": container with ID starting with 5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166 not found: ID does not exist" containerID="5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166"
Oct 30 18:47:15.691364 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.691310    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166"} err="failed to get container status \"5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166\": rpc error: code = NotFound desc = could not find container \"5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166\": container with ID starting with 5ba74488477edcbbc5112bb0f7ca652752f600abc18d7118667ed67c7c0d7166 not found: ID does not exist"
Oct 30 18:47:15.840645 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.840622    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b9db974cb-slw8w"]
Oct 30 18:47:15.888743 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:15.888706    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b9db974cb-slw8w"]
Oct 30 18:47:16.185478 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:47:16.185401    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:47:17.537334 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:17.537303    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1853522-ac6d-49a5-b86b-d8056aadc5d5" path="/var/lib/kubelet/pods/b1853522-ac6d-49a5-b86b-d8056aadc5d5/volumes"
Oct 30 18:47:20.900695 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:20.900659    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f8whs"
Oct 30 18:47:43.411914 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:43.411864    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:47:43.431194 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:43.431173    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:47:43.789427 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:47:43.789353    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:47:46.186074 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:47:46.185991    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:48:04.798201 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:04.798159    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:48:04.798854 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:04.798430    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="prometheus" containerID="cri-o://22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428" gracePeriod=600
Oct 30 18:48:04.798854 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:04.798473    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy" containerID="cri-o://c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687" gracePeriod=600
Oct 30 18:48:04.798854 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:04.798499    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy-thanos" containerID="cri-o://54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005" gracePeriod=600
Oct 30 18:48:04.798854 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:04.798537    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy-web" containerID="cri-o://6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12" gracePeriod=600
Oct 30 18:48:04.798854 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:04.798499    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="thanos-sidecar" containerID="cri-o://2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f" gracePeriod=600
Oct 30 18:48:04.798854 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:04.798602    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="config-reloader" containerID="cri-o://9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1" gracePeriod=600
Oct 30 18:48:05.035610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.035586    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:05.159212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159182    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159377 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159220    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159377 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159244    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159377 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159276    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159369    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159423    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159461    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159489    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159526    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159806 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159566    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159806 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159611    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159806 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159627    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:05.159806 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159636    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159806 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159695    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159806 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159737    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.159806 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159781    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.160192 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159833    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.160192 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159869    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.160192 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159898    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.160192 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159929    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.160192 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.159965    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle\") pod \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\" (UID: \"4a4ac65f-7a27-48a5-9bce-1fe44944d664\") "
Oct 30 18:48:05.160192 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.160174    5232 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.161058 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.161010    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:05.161598 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.161570    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:05.161928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.161900    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:05.162694 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.162625    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:05.163099 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.163052    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.163099 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.163052    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:05.163257 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.163101    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:05.163356 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.163333    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.163534 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.163510    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.164092 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.164065    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:05.164182 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.164150    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:05.164240 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.164183    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:05.164363 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.164343    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "kube-api-access-rxpvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:05.164653 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.164625    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.164948 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.164924    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.165053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.165022    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.165119 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.165081    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.165179 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.165148    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.173887 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.173859    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config" pod "4a4ac65f-7a27-48a5-9bce-1fe44944d664" (UID: "4a4ac65f-7a27-48a5-9bce-1fe44944d664"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.260908 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260866    5232 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-db\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.260908 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260896    5232 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-grpc-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.260908 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260908    5232 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260917    5232 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a4ac65f-7a27-48a5-9bce-1fe44944d664-config-out\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260928    5232 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260939    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxpvg\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-kube-api-access-rxpvg\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260948    5232 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260957    5232 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260967    5232 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-kube-rbac-proxy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260975    5232 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260983    5232 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a4ac65f-7a27-48a5-9bce-1fe44944d664-tls-assets\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.260992    5232 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.261000    5232 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-configmap-metrics-client-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.261009    5232 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.261017    5232 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.261027    5232 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-secret-metrics-client-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.261036    5232 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.261044    5232 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a4ac65f-7a27-48a5-9bce-1fe44944d664-web-config\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.261151 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.261052    5232 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a4ac65f-7a27-48a5-9bce-1fe44944d664-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.846159 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846125    5232 generic.go:356] "Generic (PLEG): container finished" podID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005" exitCode=0
Oct 30 18:48:05.846159 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846152    5232 generic.go:356] "Generic (PLEG): container finished" podID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687" exitCode=0
Oct 30 18:48:05.846159 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846158    5232 generic.go:356] "Generic (PLEG): container finished" podID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12" exitCode=0
Oct 30 18:48:05.846159 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846166    5232 generic.go:356] "Generic (PLEG): container finished" podID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f" exitCode=0
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846171    5232 generic.go:356] "Generic (PLEG): container finished" podID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1" exitCode=0
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846176    5232 generic.go:356] "Generic (PLEG): container finished" podID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428" exitCode=0
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846213    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846212    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerDied","Data":"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"}
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846293    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerDied","Data":"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"}
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846336    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerDied","Data":"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"}
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846356    5232 scope.go:122] "RemoveContainer" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846379    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerDied","Data":"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"}
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846420    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerDied","Data":"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"}
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846460    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerDied","Data":"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"}
Oct 30 18:48:05.846684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.846501    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a4ac65f-7a27-48a5-9bce-1fe44944d664","Type":"ContainerDied","Data":"737ccf945120bedf59271c77ac49cd073328e7ed96c155244050bb4edca5d3a9"}
Oct 30 18:48:05.853652 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.853629    5232 scope.go:122] "RemoveContainer" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"
Oct 30 18:48:05.860006 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.859989    5232 scope.go:122] "RemoveContainer" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"
Oct 30 18:48:05.870413 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.870385    5232 scope.go:122] "RemoveContainer" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"
Oct 30 18:48:05.876701 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.876685    5232 scope.go:122] "RemoveContainer" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"
Oct 30 18:48:05.882707 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.882691    5232 scope.go:122] "RemoveContainer" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"
Oct 30 18:48:05.889108 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.889091    5232 scope.go:122] "RemoveContainer" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"
Oct 30 18:48:05.895054 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.895037    5232 scope.go:122] "RemoveContainer" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"
Oct 30 18:48:05.895311 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:05.895294    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": container with ID starting with 54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005 not found: ID does not exist" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"
Oct 30 18:48:05.895350 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.895321    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"} err="failed to get container status \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": rpc error: code = NotFound desc = could not find container \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": container with ID starting with 54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005 not found: ID does not exist"
Oct 30 18:48:05.895350 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.895338    5232 scope.go:122] "RemoveContainer" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"
Oct 30 18:48:05.895563 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:05.895545    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": container with ID starting with c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687 not found: ID does not exist" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"
Oct 30 18:48:05.895603 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.895569    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"} err="failed to get container status \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": rpc error: code = NotFound desc = could not find container \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": container with ID starting with c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687 not found: ID does not exist"
Oct 30 18:48:05.895603 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.895587    5232 scope.go:122] "RemoveContainer" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"
Oct 30 18:48:05.895863 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:05.895842    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": container with ID starting with 6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12 not found: ID does not exist" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"
Oct 30 18:48:05.895921 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.895869    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"} err="failed to get container status \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": rpc error: code = NotFound desc = could not find container \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": container with ID starting with 6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12 not found: ID does not exist"
Oct 30 18:48:05.895921 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.895884    5232 scope.go:122] "RemoveContainer" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"
Oct 30 18:48:05.896123 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:05.896105    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": container with ID starting with 2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f not found: ID does not exist" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"
Oct 30 18:48:05.896165 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896127    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"} err="failed to get container status \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": rpc error: code = NotFound desc = could not find container \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": container with ID starting with 2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f not found: ID does not exist"
Oct 30 18:48:05.896165 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896140    5232 scope.go:122] "RemoveContainer" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"
Oct 30 18:48:05.896342 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:05.896326    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": container with ID starting with 9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1 not found: ID does not exist" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"
Oct 30 18:48:05.896376 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896345    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"} err="failed to get container status \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": rpc error: code = NotFound desc = could not find container \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": container with ID starting with 9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1 not found: ID does not exist"
Oct 30 18:48:05.896376 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896357    5232 scope.go:122] "RemoveContainer" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"
Oct 30 18:48:05.896550 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:05.896537    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": container with ID starting with 22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428 not found: ID does not exist" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"
Oct 30 18:48:05.896586 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896553    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"} err="failed to get container status \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": rpc error: code = NotFound desc = could not find container \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": container with ID starting with 22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428 not found: ID does not exist"
Oct 30 18:48:05.896586 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896565    5232 scope.go:122] "RemoveContainer" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"
Oct 30 18:48:05.896758 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:05.896744    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": container with ID starting with a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038 not found: ID does not exist" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"
Oct 30 18:48:05.896794 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896761    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"} err="failed to get container status \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": rpc error: code = NotFound desc = could not find container \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": container with ID starting with a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038 not found: ID does not exist"
Oct 30 18:48:05.896794 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896772    5232 scope.go:122] "RemoveContainer" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"
Oct 30 18:48:05.896973 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896959    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"} err="failed to get container status \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": rpc error: code = NotFound desc = could not find container \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": container with ID starting with 54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005 not found: ID does not exist"
Oct 30 18:48:05.897013 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.896973    5232 scope.go:122] "RemoveContainer" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"
Oct 30 18:48:05.897182 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.897163    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"} err="failed to get container status \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": rpc error: code = NotFound desc = could not find container \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": container with ID starting with c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687 not found: ID does not exist"
Oct 30 18:48:05.897299 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.897183    5232 scope.go:122] "RemoveContainer" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"
Oct 30 18:48:05.897430 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.897404    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"} err="failed to get container status \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": rpc error: code = NotFound desc = could not find container \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": container with ID starting with 6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12 not found: ID does not exist"
Oct 30 18:48:05.897500 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.897431    5232 scope.go:122] "RemoveContainer" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"
Oct 30 18:48:05.897628 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.897611    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"} err="failed to get container status \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": rpc error: code = NotFound desc = could not find container \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": container with ID starting with 2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f not found: ID does not exist"
Oct 30 18:48:05.897696 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.897628    5232 scope.go:122] "RemoveContainer" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"
Oct 30 18:48:05.897879 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.897858    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"} err="failed to get container status \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": rpc error: code = NotFound desc = could not find container \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": container with ID starting with 9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1 not found: ID does not exist"
Oct 30 18:48:05.897929 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.897881    5232 scope.go:122] "RemoveContainer" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"
Oct 30 18:48:05.898162 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.898139    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"} err="failed to get container status \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": rpc error: code = NotFound desc = could not find container \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": container with ID starting with 22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428 not found: ID does not exist"
Oct 30 18:48:05.898231 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.898162    5232 scope.go:122] "RemoveContainer" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"
Oct 30 18:48:05.898393 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.898374    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"} err="failed to get container status \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": rpc error: code = NotFound desc = could not find container \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": container with ID starting with a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038 not found: ID does not exist"
Oct 30 18:48:05.898448 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.898394    5232 scope.go:122] "RemoveContainer" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"
Oct 30 18:48:05.898632 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.898610    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"} err="failed to get container status \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": rpc error: code = NotFound desc = could not find container \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": container with ID starting with 54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005 not found: ID does not exist"
Oct 30 18:48:05.898696 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.898634    5232 scope.go:122] "RemoveContainer" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"
Oct 30 18:48:05.898877 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.898859    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"} err="failed to get container status \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": rpc error: code = NotFound desc = could not find container \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": container with ID starting with c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687 not found: ID does not exist"
Oct 30 18:48:05.898922 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.898877    5232 scope.go:122] "RemoveContainer" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"
Oct 30 18:48:05.899078 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899061    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"} err="failed to get container status \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": rpc error: code = NotFound desc = could not find container \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": container with ID starting with 6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12 not found: ID does not exist"
Oct 30 18:48:05.899117 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899079    5232 scope.go:122] "RemoveContainer" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"
Oct 30 18:48:05.899305 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899285    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"} err="failed to get container status \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": rpc error: code = NotFound desc = could not find container \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": container with ID starting with 2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f not found: ID does not exist"
Oct 30 18:48:05.899353 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899305    5232 scope.go:122] "RemoveContainer" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"
Oct 30 18:48:05.899517 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899500    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"} err="failed to get container status \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": rpc error: code = NotFound desc = could not find container \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": container with ID starting with 9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1 not found: ID does not exist"
Oct 30 18:48:05.899566 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899517    5232 scope.go:122] "RemoveContainer" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"
Oct 30 18:48:05.899741 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899714    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"} err="failed to get container status \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": rpc error: code = NotFound desc = could not find container \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": container with ID starting with 22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428 not found: ID does not exist"
Oct 30 18:48:05.899781 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899740    5232 scope.go:122] "RemoveContainer" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"
Oct 30 18:48:05.899968 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899952    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"} err="failed to get container status \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": rpc error: code = NotFound desc = could not find container \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": container with ID starting with a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038 not found: ID does not exist"
Oct 30 18:48:05.900013 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.899969    5232 scope.go:122] "RemoveContainer" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"
Oct 30 18:48:05.900182 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900166    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"} err="failed to get container status \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": rpc error: code = NotFound desc = could not find container \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": container with ID starting with 54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005 not found: ID does not exist"
Oct 30 18:48:05.900225 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900183    5232 scope.go:122] "RemoveContainer" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"
Oct 30 18:48:05.900375 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900359    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"} err="failed to get container status \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": rpc error: code = NotFound desc = could not find container \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": container with ID starting with c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687 not found: ID does not exist"
Oct 30 18:48:05.900415 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900374    5232 scope.go:122] "RemoveContainer" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"
Oct 30 18:48:05.900594 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900576    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"} err="failed to get container status \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": rpc error: code = NotFound desc = could not find container \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": container with ID starting with 6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12 not found: ID does not exist"
Oct 30 18:48:05.900641 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900596    5232 scope.go:122] "RemoveContainer" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"
Oct 30 18:48:05.900774 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900757    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"} err="failed to get container status \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": rpc error: code = NotFound desc = could not find container \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": container with ID starting with 2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f not found: ID does not exist"
Oct 30 18:48:05.900829 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900776    5232 scope.go:122] "RemoveContainer" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"
Oct 30 18:48:05.901007 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.900987    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"} err="failed to get container status \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": rpc error: code = NotFound desc = could not find container \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": container with ID starting with 9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1 not found: ID does not exist"
Oct 30 18:48:05.901070 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901008    5232 scope.go:122] "RemoveContainer" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"
Oct 30 18:48:05.901234 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901215    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"} err="failed to get container status \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": rpc error: code = NotFound desc = could not find container \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": container with ID starting with 22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428 not found: ID does not exist"
Oct 30 18:48:05.901311 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901235    5232 scope.go:122] "RemoveContainer" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"
Oct 30 18:48:05.901437 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901415    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"} err="failed to get container status \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": rpc error: code = NotFound desc = could not find container \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": container with ID starting with a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038 not found: ID does not exist"
Oct 30 18:48:05.901481 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901437    5232 scope.go:122] "RemoveContainer" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"
Oct 30 18:48:05.901629 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901611    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"} err="failed to get container status \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": rpc error: code = NotFound desc = could not find container \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": container with ID starting with 54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005 not found: ID does not exist"
Oct 30 18:48:05.901693 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901630    5232 scope.go:122] "RemoveContainer" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"
Oct 30 18:48:05.901849 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901831    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"} err="failed to get container status \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": rpc error: code = NotFound desc = could not find container \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": container with ID starting with c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687 not found: ID does not exist"
Oct 30 18:48:05.901897 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.901850    5232 scope.go:122] "RemoveContainer" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"
Oct 30 18:48:05.902040 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902023    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"} err="failed to get container status \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": rpc error: code = NotFound desc = could not find container \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": container with ID starting with 6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12 not found: ID does not exist"
Oct 30 18:48:05.902106 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902041    5232 scope.go:122] "RemoveContainer" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"
Oct 30 18:48:05.902265 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902247    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"} err="failed to get container status \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": rpc error: code = NotFound desc = could not find container \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": container with ID starting with 2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f not found: ID does not exist"
Oct 30 18:48:05.902306 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902266    5232 scope.go:122] "RemoveContainer" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"
Oct 30 18:48:05.902455 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902436    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"} err="failed to get container status \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": rpc error: code = NotFound desc = could not find container \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": container with ID starting with 9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1 not found: ID does not exist"
Oct 30 18:48:05.902494 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902456    5232 scope.go:122] "RemoveContainer" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"
Oct 30 18:48:05.902647 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902632    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"} err="failed to get container status \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": rpc error: code = NotFound desc = could not find container \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": container with ID starting with 22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428 not found: ID does not exist"
Oct 30 18:48:05.902712 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902648    5232 scope.go:122] "RemoveContainer" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"
Oct 30 18:48:05.902860 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902846    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"} err="failed to get container status \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": rpc error: code = NotFound desc = could not find container \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": container with ID starting with a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038 not found: ID does not exist"
Oct 30 18:48:05.902908 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.902861    5232 scope.go:122] "RemoveContainer" containerID="54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"
Oct 30 18:48:05.903042 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903026    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005"} err="failed to get container status \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": rpc error: code = NotFound desc = could not find container \"54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005\": container with ID starting with 54610af2d16661d64f97b26a5e64d2945cbabc76b2ebedea800ea77f2b6a0005 not found: ID does not exist"
Oct 30 18:48:05.903087 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903043    5232 scope.go:122] "RemoveContainer" containerID="c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"
Oct 30 18:48:05.903236 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903218    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687"} err="failed to get container status \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": rpc error: code = NotFound desc = could not find container \"c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687\": container with ID starting with c1d2817abbf321c1c61f2da38420785dd0f0f1f45b3eb69e0e725c0164af3687 not found: ID does not exist"
Oct 30 18:48:05.903304 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903236    5232 scope.go:122] "RemoveContainer" containerID="6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"
Oct 30 18:48:05.903451 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903430    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12"} err="failed to get container status \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": rpc error: code = NotFound desc = could not find container \"6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12\": container with ID starting with 6d76e8bce041bd3485d030634934453e0297c22caaefb587c2103f95c0138b12 not found: ID does not exist"
Oct 30 18:48:05.903495 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903452    5232 scope.go:122] "RemoveContainer" containerID="2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"
Oct 30 18:48:05.903642 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903625    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f"} err="failed to get container status \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": rpc error: code = NotFound desc = could not find container \"2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f\": container with ID starting with 2f0eab8b65f7d254c1f974482fe639d136e5c23f4848c70d660f693342ba0a0f not found: ID does not exist"
Oct 30 18:48:05.903709 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903643    5232 scope.go:122] "RemoveContainer" containerID="9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"
Oct 30 18:48:05.903945 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903928    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1"} err="failed to get container status \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": rpc error: code = NotFound desc = could not find container \"9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1\": container with ID starting with 9cd2527e175f01ca9ce6adb6a065fa66935db2328826c795e781aec2292b45c1 not found: ID does not exist"
Oct 30 18:48:05.904002 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.903946    5232 scope.go:122] "RemoveContainer" containerID="22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"
Oct 30 18:48:05.904147 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.904126    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428"} err="failed to get container status \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": rpc error: code = NotFound desc = could not find container \"22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428\": container with ID starting with 22857dec73ecca76c0df86f5780fa0afd627ced7eea3c0525c9472296164e428 not found: ID does not exist"
Oct 30 18:48:05.904212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.904147    5232 scope.go:122] "RemoveContainer" containerID="a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"
Oct 30 18:48:05.904372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:05.904351    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038"} err="failed to get container status \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": rpc error: code = NotFound desc = could not find container \"a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038\": container with ID starting with a8b81ce79ad921b55cdacde37b69b5b92dbae6bf477468f59662633a9796f038 not found: ID does not exist"
Oct 30 18:48:06.007718 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.007685    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:48:06.055351 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055317    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:48:06.055622 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055602    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:48:06.055883 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055866    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="init-config-reloader"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055885    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="init-config-reloader"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055898    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="config-reloader"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055908    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="config-reloader"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055917    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy-web"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055924    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy-web"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055934    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055941    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055955    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="thanos-sidecar"
Oct 30 18:48:06.055960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055962    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="thanos-sidecar"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055971    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy-thanos"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055978    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy-thanos"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.055995    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="prometheus"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056002    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="prometheus"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056015    5232 cpu_manager.go:414] "containerMap: removing container" podUID="b1853522-ac6d-49a5-b86b-d8056aadc5d5" containerName="console"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056021    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1853522-ac6d-49a5-b86b-d8056aadc5d5" containerName="console"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056091    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1853522-ac6d-49a5-b86b-d8056aadc5d5" containerName="console"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056105    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="config-reloader"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056113    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy-web"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056122    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056130    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="thanos-sidecar"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056137    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="kube-rbac-proxy-thanos"
Oct 30 18:48:06.056372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.056147    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" containerName="prometheus"
Oct 30 18:48:06.060467 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.060447    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.112619 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.112584    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:48:06.159265 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159224    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 18:48:06.159450 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159323    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 18:48:06.159618 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159546    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 18:48:06.159618 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159556    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 18:48:06.159618 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159575    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 18:48:06.159618 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159603    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 18:48:06.159618 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159609    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 18:48:06.159888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159623    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 18:48:06.159888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159609    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 18:48:06.159888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.159742    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 18:48:06.160195 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.160180    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7tn3b1m8racr8\""
Oct 30 18:48:06.160892 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.160873    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 18:48:06.160892 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.160881    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6zff9\""
Oct 30 18:48:06.161546 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.161519    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 18:48:06.161546 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.161546    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 18:48:06.161721 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.161666    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 18:48:06.168859 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.168842    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.168929 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.168867    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.168929 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.168884    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.168929 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.168903    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169062 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.168971    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169095 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169066    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169095 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169088    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169168 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169105    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-config\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169168 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169133    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3179b985-7263-403d-b791-666db397de1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169228 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169172    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169228 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169211    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169246    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169324 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169288    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2ss9\" (UniqueName: \"kubernetes.io/projected/3179b985-7263-403d-b791-666db397de1b-kube-api-access-f2ss9\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169353 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169343    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3179b985-7263-403d-b791-666db397de1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169361    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169420 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169402    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169450 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169431    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169513 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169453    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169513 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169505    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.169585 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.169523    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270573 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270538    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270573 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270574    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270594    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270613    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270636    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270662    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270688    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270711    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-config\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270743    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3179b985-7263-403d-b791-666db397de1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270760    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270776    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270790    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.270839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270833    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2ss9\" (UniqueName: \"kubernetes.io/projected/3179b985-7263-403d-b791-666db397de1b-kube-api-access-f2ss9\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270868    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3179b985-7263-403d-b791-666db397de1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270889    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270913    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270943    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.270976    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.271022    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.271048    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.271114    5232 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:48:06.271804 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.271439    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271804 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.271479    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.271804 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.271566    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.272905 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.272361    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.272905 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.272427    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.272905 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.272584    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.273495 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.273133    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.274569 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.274542    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.274934 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.274888    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3179b985-7263-403d-b791-666db397de1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.275321 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.275279    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.275925 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.275894    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.276030 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.275951    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.276030 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.275956    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-config\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.276276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.276237    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.276276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.276265    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.276420 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.276373    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.276420 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.276410    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3179b985-7263-403d-b791-666db397de1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.276751 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.276733    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3179b985-7263-403d-b791-666db397de1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.277327 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.277311    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3179b985-7263-403d-b791-666db397de1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.302666 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.302642    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:48:06.332763 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.332696    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2ss9\" (UniqueName: \"kubernetes.io/projected/3179b985-7263-403d-b791-666db397de1b-kube-api-access-f2ss9\") pod \"prometheus-k8s-0\" (UID: \"3179b985-7263-403d-b791-666db397de1b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.382634 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.382601    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:06.812905 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.812874    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:48:06.819228 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:48:06.819197    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3179b985_7263_403d_b791_666db397de1b.slice/crio-b35324837c2e72442e6c7b7cb3327433f2f92864cf9bcd14bbe6c804abb27a63 WatchSource:0}: Error finding container b35324837c2e72442e6c7b7cb3327433f2f92864cf9bcd14bbe6c804abb27a63: Status 404 returned error can't find the container with id b35324837c2e72442e6c7b7cb3327433f2f92864cf9bcd14bbe6c804abb27a63
Oct 30 18:48:06.851187 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:06.851164    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerStarted","Data":"b35324837c2e72442e6c7b7cb3327433f2f92864cf9bcd14bbe6c804abb27a63"}
Oct 30 18:48:07.537470 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:07.537432    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a4ac65f-7a27-48a5-9bce-1fe44944d664" path="/var/lib/kubelet/pods/4a4ac65f-7a27-48a5-9bce-1fe44944d664/volumes"
Oct 30 18:48:07.855073 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:07.855039    5232 generic.go:356] "Generic (PLEG): container finished" podID="3179b985-7263-403d-b791-666db397de1b" containerID="69ee7a019c167e4ea21c1aff1f882f2fec1879b3dd4918303ee816474c595e8e" exitCode=0
Oct 30 18:48:07.855435 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:07.855130    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerDied","Data":"69ee7a019c167e4ea21c1aff1f882f2fec1879b3dd4918303ee816474c595e8e"}
Oct 30 18:48:08.861730 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.861702    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 18:48:08.862158 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.862081    5232 generic.go:356] "Generic (PLEG): container finished" podID="3179b985-7263-403d-b791-666db397de1b" containerID="85ff0c720099f5ba275c3b5cbbde704c83acb3fb3073cbd6bcc27556412e077f" exitCode=2
Oct 30 18:48:08.862158 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.862118    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerStarted","Data":"b870f134db2bbf99da15933cd66af1f2c15f60d7b89b3f4f5c9c0631ab751bbd"}
Oct 30 18:48:08.862227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.862176    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerStarted","Data":"b5c3b7b8390a481348781a0d9c5b65f8416e99bd88dbaa8f7017defc32036fdd"}
Oct 30 18:48:08.862227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.862204    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerStarted","Data":"49129d9d57091a4dde52b379e6c0ee823702c95fe446617d912c26a3108b5e2f"}
Oct 30 18:48:08.862294 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.862227    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerStarted","Data":"e2818afafcab182d42370f9777b4265e2b276a4956c28ff774c81cafbe7e8eed"}
Oct 30 18:48:08.862294 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.862259    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerStarted","Data":"bebb1ee14a34653332e4f2d5d9ded12db7d0665f17fc1eb87fb51dd9f438aeea"}
Oct 30 18:48:08.862294 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.862286    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerDied","Data":"85ff0c720099f5ba275c3b5cbbde704c83acb3fb3073cbd6bcc27556412e077f"}
Oct 30 18:48:08.862532 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:08.862519    5232 scope.go:122] "RemoveContainer" containerID="85ff0c720099f5ba275c3b5cbbde704c83acb3fb3073cbd6bcc27556412e077f"
Oct 30 18:48:09.871923 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:09.871893    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 18:48:09.872348 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:09.872301    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3179b985-7263-403d-b791-666db397de1b","Type":"ContainerStarted","Data":"3c19acddc2386c4264661bef3c9b733589a840b01bd37e9ed33399a60f432331"}
Oct 30 18:48:10.149030 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:10.148919    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.132476132 podStartE2EDuration="5.148906067s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.429935ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:10.148688465 +0000 UTC m=+82.733171465" watchObservedRunningTime="2026-10-30 18:48:10.148906067 +0000 UTC m=+82.733389064"
Oct 30 18:48:11.383593 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:11.383536    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:48:16.185321 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:16.185280    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:48:17.682047 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:17.682020    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f888ff8d9-78xgg"]
Oct 30 18:48:42.700687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.700578    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5f888ff8d9-78xgg" podUID="731eee3b-34bd-4188-9c74-a6a684acce6b" containerName="console" containerID="cri-o://75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e" gracePeriod=15
Oct 30 18:48:42.954542 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.954488    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:48:42.979338 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.979314    5232 generic.go:356] "Generic (PLEG): container finished" podID="731eee3b-34bd-4188-9c74-a6a684acce6b" containerID="75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e" exitCode=0
Oct 30 18:48:42.979446 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.979355    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f888ff8d9-78xgg"
Oct 30 18:48:42.979446 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.979368    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f888ff8d9-78xgg" event={"ID":"731eee3b-34bd-4188-9c74-a6a684acce6b","Type":"ContainerDied","Data":"75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e"}
Oct 30 18:48:42.979446 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.979408    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f888ff8d9-78xgg" event={"ID":"731eee3b-34bd-4188-9c74-a6a684acce6b","Type":"ContainerDied","Data":"b8c05cd7abffae36b8d4f9dbfc12e164f19673eb786383f1486e38c2f25feb0b"}
Oct 30 18:48:42.979446 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.979427    5232 scope.go:122] "RemoveContainer" containerID="75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e"
Oct 30 18:48:42.986586 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.986570    5232 scope.go:122] "RemoveContainer" containerID="75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e"
Oct 30 18:48:42.986852 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:42.986833    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e\": container with ID starting with 75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e not found: ID does not exist" containerID="75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e"
Oct 30 18:48:42.986930 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:42.986860    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e"} err="failed to get container status \"75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e\": rpc error: code = NotFound desc = could not find container \"75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e\": container with ID starting with 75b0a46b6c487c34bd80708083848767462a7e58a3f2ab20a6be6c5ba09cfe3e not found: ID does not exist"
Oct 30 18:48:43.037560 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.037529    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config\") pod \"731eee3b-34bd-4188-9c74-a6a684acce6b\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") "
Oct 30 18:48:43.037687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.037569    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca\") pod \"731eee3b-34bd-4188-9c74-a6a684acce6b\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") "
Oct 30 18:48:43.037687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.037595    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq\" (UniqueName: \"kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq\") pod \"731eee3b-34bd-4188-9c74-a6a684acce6b\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") "
Oct 30 18:48:43.037687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.037620    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert\") pod \"731eee3b-34bd-4188-9c74-a6a684acce6b\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") "
Oct 30 18:48:43.037687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.037643    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle\") pod \"731eee3b-34bd-4188-9c74-a6a684acce6b\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") "
Oct 30 18:48:43.037687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.037685    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp\" (UniqueName: \"kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp\") pod \"731eee3b-34bd-4188-9c74-a6a684acce6b\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") "
Oct 30 18:48:43.037958 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.037708    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config\") pod \"731eee3b-34bd-4188-9c74-a6a684acce6b\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") "
Oct 30 18:48:43.037958 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.037741    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert\") pod \"731eee3b-34bd-4188-9c74-a6a684acce6b\" (UID: \"731eee3b-34bd-4188-9c74-a6a684acce6b\") "
Oct 30 18:48:43.038077 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.038045    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca" pod "731eee3b-34bd-4188-9c74-a6a684acce6b" (UID: "731eee3b-34bd-4188-9c74-a6a684acce6b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:43.038221 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.038195    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle" pod "731eee3b-34bd-4188-9c74-a6a684acce6b" (UID: "731eee3b-34bd-4188-9c74-a6a684acce6b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:43.038221 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.038199    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp" pod "731eee3b-34bd-4188-9c74-a6a684acce6b" (UID: "731eee3b-34bd-4188-9c74-a6a684acce6b"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:43.038362 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.038295    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config" pod "731eee3b-34bd-4188-9c74-a6a684acce6b" (UID: "731eee3b-34bd-4188-9c74-a6a684acce6b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:43.038362 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.038293    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert" pod "731eee3b-34bd-4188-9c74-a6a684acce6b" (UID: "731eee3b-34bd-4188-9c74-a6a684acce6b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:43.039903 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.039877    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq" pod "731eee3b-34bd-4188-9c74-a6a684acce6b" (UID: "731eee3b-34bd-4188-9c74-a6a684acce6b"). InnerVolumeSpecName "kube-api-access-d5nrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:43.040192 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.040167    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config" pod "731eee3b-34bd-4188-9c74-a6a684acce6b" (UID: "731eee3b-34bd-4188-9c74-a6a684acce6b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:43.040271 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.040206    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert" pod "731eee3b-34bd-4188-9c74-a6a684acce6b" (UID: "731eee3b-34bd-4188-9c74-a6a684acce6b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:43.138791 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.138758    5232 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-oauth-config\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:43.138791 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.138786    5232 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:43.138791 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.138796    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5nrq\" (UniqueName: \"kubernetes.io/projected/731eee3b-34bd-4188-9c74-a6a684acce6b-kube-api-access-d5nrq\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:43.139040 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.138807    5232 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/731eee3b-34bd-4188-9c74-a6a684acce6b-console-serving-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:43.139040 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.138834    5232 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-trusted-ca-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:43.139040 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.138844    5232 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731eee3b-34bd-4188-9c74-a6a684acce6b-tmp\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:43.139040 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.138854    5232 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-console-config\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:43.139040 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.138863    5232 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/731eee3b-34bd-4188-9c74-a6a684acce6b-oauth-serving-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:43.394546 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.394513    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f888ff8d9-78xgg"]
Oct 30 18:48:43.396561 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.396535    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f888ff8d9-78xgg"]
Oct 30 18:48:43.537182 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:48:43.537150    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="731eee3b-34bd-4188-9c74-a6a684acce6b" path="/var/lib/kubelet/pods/731eee3b-34bd-4188-9c74-a6a684acce6b/volumes"
Oct 30 18:48:46.184881 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:48:46.184839    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:49:06.383636 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:49:06.383585    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:49:06.398657 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:49:06.398631    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:49:07.068214 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:49:07.068182    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:49:16.185792 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:49:16.185752    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:49:46.185678 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:49:46.185644    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:50:16.185424 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:50:16.185389    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:50:46.185954 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:50:46.185920    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:51:16.184903 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:51:16.184866    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:51:46.185434 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:51:46.185354    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:51:47.393455 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.393412    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"]
Oct 30 18:51:47.393984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.393789    5232 cpu_manager.go:414] "containerMap: removing container" podUID="731eee3b-34bd-4188-9c74-a6a684acce6b" containerName="console"
Oct 30 18:51:47.393984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.393807    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="731eee3b-34bd-4188-9c74-a6a684acce6b" containerName="console"
Oct 30 18:51:47.393984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.393902    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="731eee3b-34bd-4188-9c74-a6a684acce6b" containerName="console"
Oct 30 18:51:47.396241 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.396215    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.446534 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.446506    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 18:51:47.494021 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.493992    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 18:51:47.494189 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.494070    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-k9pq7\""
Oct 30 18:51:47.494628 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.494610    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"]
Oct 30 18:51:47.494984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.494968    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"]
Oct 30 18:51:47.497200 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.497186    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.528912 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.528887    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 18:51:47.532922 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.532899    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 18:51:47.539854 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.539833    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dvvc\" (UniqueName: \"kubernetes.io/projected/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-kube-api-access-6dvvc\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.539991 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.539861    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.539991 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.539979    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.554274 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.554252    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-g94zr\""
Oct 30 18:51:47.570067 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.570046    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"]
Oct 30 18:51:47.585438 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.585416    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"]
Oct 30 18:51:47.587880 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.587858    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.637689 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.637658    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-lcb68\""
Oct 30 18:51:47.640936 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.640916    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.641002 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.640951    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dvvc\" (UniqueName: \"kubernetes.io/projected/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-kube-api-access-6dvvc\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.641039 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.641001    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.641083 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.641048    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.641124 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.641082    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.641174 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.641131    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvcmm\" (UniqueName: \"kubernetes.io/projected/64796a8b-7b79-46f3-b84e-306f803430d9-kube-api-access-xvcmm\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.641406 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.641388    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.641474 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.641442    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.648186 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.648141    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"]
Oct 30 18:51:47.677182 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.677155    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"]
Oct 30 18:51:47.679654 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.679630    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.683419 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.683398    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"]
Oct 30 18:51:47.685830 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.685787    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:47.730081 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.730055    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-sbsgn\""
Oct 30 18:51:47.735736 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.735725    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-wdxrq\""
Oct 30 18:51:47.739841 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.739800    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"]
Oct 30 18:51:47.741432 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.741415    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvcmm\" (UniqueName: \"kubernetes.io/projected/64796a8b-7b79-46f3-b84e-306f803430d9-kube-api-access-xvcmm\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.741481 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.741451    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.741481 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.741475    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.741573 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.741491    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.741627 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.741575    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.741687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.741630    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnn48\" (UniqueName: \"kubernetes.io/projected/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-kube-api-access-wnn48\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.741925 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.741904    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.741925 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.741919    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.746521 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.746499    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 18:51:47.746937 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.746920    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"]
Oct 30 18:51:47.751157 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.751134    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dvvc\" (UniqueName: \"kubernetes.io/projected/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-kube-api-access-6dvvc\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:47.800072 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.800036    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvcmm\" (UniqueName: \"kubernetes.io/projected/64796a8b-7b79-46f3-b84e-306f803430d9-kube-api-access-xvcmm\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.809773 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.809747    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:51:47.842907 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.842859    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.842907 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.842898    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9gpv\" (UniqueName: \"kubernetes.io/projected/97d13881-9356-4a5b-8809-58262035eb5e-kube-api-access-x9gpv\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.843163 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.842918    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smbpd\" (UniqueName: \"kubernetes.io/projected/f33f719b-232d-4393-92e4-282176c11a7d-kube-api-access-smbpd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:47.843163 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.843020    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnn48\" (UniqueName: \"kubernetes.io/projected/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-kube-api-access-wnn48\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.843163 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.843042    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:47.843163 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.843073    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:47.843367 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.843158    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.843367 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.843199    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.843367 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.843234    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.843567 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.843548    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.843626 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.843575    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.903020 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.902948    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnn48\" (UniqueName: \"kubernetes.io/projected/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-kube-api-access-wnn48\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:47.944574 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.944533    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.944746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.944584    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.944746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.944601    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gpv\" (UniqueName: \"kubernetes.io/projected/97d13881-9356-4a5b-8809-58262035eb5e-kube-api-access-x9gpv\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.944746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.944620    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smbpd\" (UniqueName: \"kubernetes.io/projected/f33f719b-232d-4393-92e4-282176c11a7d-kube-api-access-smbpd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:47.944746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.944648    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:47.944746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.944684    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:47.945005 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.944989    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.945102 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.945080    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:47.945140 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.945101    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:47.945140 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:47.945105    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:48.007792 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.007759    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smbpd\" (UniqueName: \"kubernetes.io/projected/f33f719b-232d-4393-92e4-282176c11a7d-kube-api-access-smbpd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:48.007943 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.007759    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9gpv\" (UniqueName: \"kubernetes.io/projected/97d13881-9356-4a5b-8809-58262035eb5e-kube-api-access-x9gpv\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:48.009373 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.009355    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:51:48.183287 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.183207    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"]
Oct 30 18:51:48.190022 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:48.189990    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64796a8b_7b79_46f3_b84e_306f803430d9.slice/crio-401247afc65dab623f085bdb1346af680b52224d735742a3c061e485fedbd98f WatchSource:0}: Error finding container 401247afc65dab623f085bdb1346af680b52224d735742a3c061e485fedbd98f: Status 404 returned error can't find the container with id 401247afc65dab623f085bdb1346af680b52224d735742a3c061e485fedbd98f
Oct 30 18:51:48.200613 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.200591    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:51:48.296025 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.295993    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:51:48.302612 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.302546    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:51:48.374691 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.374666    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"]
Oct 30 18:51:48.382356 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:48.382331    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca19d4b3_5ea8_412e_b0d2_a0fc3b3f01a7.slice/crio-e98dab2b764ddba946fbec4deedbaa81b05bfdc1a66c2d8442f4e5644e7d580f WatchSource:0}: Error finding container e98dab2b764ddba946fbec4deedbaa81b05bfdc1a66c2d8442f4e5644e7d580f: Status 404 returned error can't find the container with id e98dab2b764ddba946fbec4deedbaa81b05bfdc1a66c2d8442f4e5644e7d580f
Oct 30 18:51:48.552713 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.552638    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp" event={"ID":"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7","Type":"ContainerStarted","Data":"e98dab2b764ddba946fbec4deedbaa81b05bfdc1a66c2d8442f4e5644e7d580f"}
Oct 30 18:51:48.553574 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.553555    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl" event={"ID":"64796a8b-7b79-46f3-b84e-306f803430d9","Type":"ContainerStarted","Data":"401247afc65dab623f085bdb1346af680b52224d735742a3c061e485fedbd98f"}
Oct 30 18:51:48.565507 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.565483    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"]
Oct 30 18:51:48.571986 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:48.571960    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8e28d4_3c32_4801_8fa8_4a19a59fdb14.slice/crio-4cc8d3e9154e0fa00f39808c341e55a703f030412be9ab524bee7e8dee3a2e4f WatchSource:0}: Error finding container 4cc8d3e9154e0fa00f39808c341e55a703f030412be9ab524bee7e8dee3a2e4f: Status 404 returned error can't find the container with id 4cc8d3e9154e0fa00f39808c341e55a703f030412be9ab524bee7e8dee3a2e4f
Oct 30 18:51:48.665476 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.665442    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"]
Oct 30 18:51:48.672073 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:48.672047    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d13881_9356_4a5b_8809_58262035eb5e.slice/crio-abb22a252edef9c7bb72b2e220cb6b4c77625522192e36ec73457d4d7e14b9af WatchSource:0}: Error finding container abb22a252edef9c7bb72b2e220cb6b4c77625522192e36ec73457d4d7e14b9af: Status 404 returned error can't find the container with id abb22a252edef9c7bb72b2e220cb6b4c77625522192e36ec73457d4d7e14b9af
Oct 30 18:51:48.673274 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:48.673256    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"]
Oct 30 18:51:48.680023 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:48.679996    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33f719b_232d_4393_92e4_282176c11a7d.slice/crio-4493501c21b33b2a6411fe9010d3fc9322a207b83604d84851e91fc8580dd7b8 WatchSource:0}: Error finding container 4493501c21b33b2a6411fe9010d3fc9322a207b83604d84851e91fc8580dd7b8: Status 404 returned error can't find the container with id 4493501c21b33b2a6411fe9010d3fc9322a207b83604d84851e91fc8580dd7b8
Oct 30 18:51:49.296772 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.296740    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"]
Oct 30 18:51:49.299901 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.299872    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.350803 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.350768    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-fngx5\""
Oct 30 18:51:49.359647 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.359563    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"]
Oct 30 18:51:49.457917 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.457885    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdrcx\" (UniqueName: \"kubernetes.io/projected/686bde4e-07e4-4381-9028-05db35bf4509-kube-api-access-kdrcx\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.458099 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.457943    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.458172 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.458085    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.558805 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.558724    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.558805 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.558776    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.559285 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.558862    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdrcx\" (UniqueName: \"kubernetes.io/projected/686bde4e-07e4-4381-9028-05db35bf4509-kube-api-access-kdrcx\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.559285 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.559230    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.559411 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.559389    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.561197 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.561165    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt" event={"ID":"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14","Type":"ContainerStarted","Data":"4cc8d3e9154e0fa00f39808c341e55a703f030412be9ab524bee7e8dee3a2e4f"}
Oct 30 18:51:49.563419 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.563390    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph" event={"ID":"f33f719b-232d-4393-92e4-282176c11a7d","Type":"ContainerStarted","Data":"4493501c21b33b2a6411fe9010d3fc9322a207b83604d84851e91fc8580dd7b8"}
Oct 30 18:51:49.565074 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.565044    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n" event={"ID":"97d13881-9356-4a5b-8809-58262035eb5e","Type":"ContainerStarted","Data":"abb22a252edef9c7bb72b2e220cb6b4c77625522192e36ec73457d4d7e14b9af"}
Oct 30 18:51:49.621290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.621258    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdrcx\" (UniqueName: \"kubernetes.io/projected/686bde4e-07e4-4381-9028-05db35bf4509-kube-api-access-kdrcx\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:49.918958 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:49.918918    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:51:50.319164 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:50.319089    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"]
Oct 30 18:51:50.334127 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:50.328849    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686bde4e_07e4_4381_9028_05db35bf4509.slice/crio-e7c9b59f701ae6ece73fe541442d8b0603d2a1095ea02e201ee2c83b7da5642b WatchSource:0}: Error finding container e7c9b59f701ae6ece73fe541442d8b0603d2a1095ea02e201ee2c83b7da5642b: Status 404 returned error can't find the container with id e7c9b59f701ae6ece73fe541442d8b0603d2a1095ea02e201ee2c83b7da5642b
Oct 30 18:51:50.334127 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:50.332387    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:51:50.572746 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:50.572658    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l" event={"ID":"686bde4e-07e4-4381-9028-05db35bf4509","Type":"ContainerStarted","Data":"e7c9b59f701ae6ece73fe541442d8b0603d2a1095ea02e201ee2c83b7da5642b"}
Oct 30 18:51:51.917704 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:51.915998    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"]
Oct 30 18:51:51.921552 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:51.921525    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.031467 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.031430    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-xtspv\""
Oct 30 18:51:52.031901 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.031860    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"]
Oct 30 18:51:52.080198 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.080151    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjv2m\" (UniqueName: \"kubernetes.io/projected/ca4d2130-307e-45f9-87b7-9621d7eb12d9-kube-api-access-rjv2m\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.080391 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.080216    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.080391 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.080341    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.181254 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.181165    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.181436 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.181259    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.181436 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.181331    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjv2m\" (UniqueName: \"kubernetes.io/projected/ca4d2130-307e-45f9-87b7-9621d7eb12d9-kube-api-access-rjv2m\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.182336 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.182274    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.182653 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.182620    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.247134 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.247069    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjv2m\" (UniqueName: \"kubernetes.io/projected/ca4d2130-307e-45f9-87b7-9621d7eb12d9-kube-api-access-rjv2m\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.547228 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.547154    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:51:52.967630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:52.967542    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"]
Oct 30 18:51:53.322552 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.321208    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"]
Oct 30 18:51:53.330450 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.326301    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.381321 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.381098    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:51:53.381321 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.381222    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:51:53.381321 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.381290    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 18:51:53.400736 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.400626    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"]
Oct 30 18:51:53.427767 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.427694    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4d7rj\""
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495398    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495452    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495489    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495524    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495592    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495627    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495672    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fphhs\" (UniqueName: \"kubernetes.io/projected/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-kube-api-access-fphhs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495707    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.495979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.495740    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596431    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fphhs\" (UniqueName: \"kubernetes.io/projected/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-kube-api-access-fphhs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596503    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596536    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596585    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596619    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596656    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596684    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596756    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.596931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.596793    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.597607 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.597160    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.597607 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.597401    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.598828 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.598153    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.598828 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.598341    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.598828 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.598470    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.600589 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.600507    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n" event={"ID":"ca4d2130-307e-45f9-87b7-9621d7eb12d9","Type":"ContainerStarted","Data":"5747392234b94bfe16d165fe5cdd19a5e8adb255f9ff450550a90313ec89d6c8"}
Oct 30 18:51:53.601496 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.601437    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.607480 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.607432    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.664665 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.664631    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.666720 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.666627    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fphhs\" (UniqueName: \"kubernetes.io/projected/ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a-kube-api-access-fphhs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-gshcc\" (UID: \"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:53.946303 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:53.946201    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:51:54.446533 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:54.446496    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"]
Oct 30 18:51:54.458759 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:54.458722    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddbcb069_1f08_4ba2_bc8c_14cc6f40d90a.slice/crio-dbb43da409fbf5a373f9e2f5ad0e4e621b26f86fc64ee45c483718e291e917ce WatchSource:0}: Error finding container dbb43da409fbf5a373f9e2f5ad0e4e621b26f86fc64ee45c483718e291e917ce: Status 404 returned error can't find the container with id dbb43da409fbf5a373f9e2f5ad0e4e621b26f86fc64ee45c483718e291e917ce
Oct 30 18:51:54.608935 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:54.608897    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" event={"ID":"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a","Type":"ContainerStarted","Data":"dbb43da409fbf5a373f9e2f5ad0e4e621b26f86fc64ee45c483718e291e917ce"}
Oct 30 18:51:55.122863 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.122804    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"]
Oct 30 18:51:55.127798 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.127215    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.156710 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.155111    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"]
Oct 30 18:51:55.159206 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.158849    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.203299 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.203240    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"]
Oct 30 18:51:55.204465 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.204442    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"]
Oct 30 18:51:55.208661 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.208247    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.230125 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.230093    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"]
Oct 30 18:51:55.280490 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.280457    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"]
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310303    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6hmj\" (UniqueName: \"kubernetes.io/projected/a1632c13-8535-47c3-a331-c8dbc9141b3a-kube-api-access-p6hmj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310354    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310398    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j95sc\" (UniqueName: \"kubernetes.io/projected/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-kube-api-access-j95sc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310437    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkzp7\" (UniqueName: \"kubernetes.io/projected/94081c24-324d-4214-9659-0e0628f7ad8d-kube-api-access-kkzp7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310494    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310525    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310561    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310588    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.310851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.310616    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.411890    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j95sc\" (UniqueName: \"kubernetes.io/projected/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-kube-api-access-j95sc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.411967    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkzp7\" (UniqueName: \"kubernetes.io/projected/94081c24-324d-4214-9659-0e0628f7ad8d-kube-api-access-kkzp7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.412025    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.412055    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.412096    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.412125    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.412171    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.412218    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hmj\" (UniqueName: \"kubernetes.io/projected/a1632c13-8535-47c3-a331-c8dbc9141b3a-kube-api-access-p6hmj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.412249    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.412687    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.413175    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.413274    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.413457    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.413833 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.413721    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.414534 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.414221    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.489553 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.489512    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkzp7\" (UniqueName: \"kubernetes.io/projected/94081c24-324d-4214-9659-0e0628f7ad8d-kube-api-access-kkzp7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.490029 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.489564    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j95sc\" (UniqueName: \"kubernetes.io/projected/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-kube-api-access-j95sc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.490029 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.489512    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6hmj\" (UniqueName: \"kubernetes.io/projected/a1632c13-8535-47c3-a331-c8dbc9141b3a-kube-api-access-p6hmj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.530521 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.530223    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:51:55.749447 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.749132    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:51:55.779766 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.779724    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:51:55.928432 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:55.928397    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"]
Oct 30 18:51:55.955073 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:55.955037    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e42e57f_5a8b_4e1b_9db8_78367840f5f3.slice/crio-f979c66ac13599bc91b41629d8e8327a905d8b5109390364ef0fc244bb294056 WatchSource:0}: Error finding container f979c66ac13599bc91b41629d8e8327a905d8b5109390364ef0fc244bb294056: Status 404 returned error can't find the container with id f979c66ac13599bc91b41629d8e8327a905d8b5109390364ef0fc244bb294056
Oct 30 18:51:56.153939 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:56.153904    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"]
Oct 30 18:51:56.163856 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:51:56.163824    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1632c13_8535_47c3_a331_c8dbc9141b3a.slice/crio-a5eea323bd59f0d31683c03bdb690dffdf7f2a927223777f6b3e0912cc83f307 WatchSource:0}: Error finding container a5eea323bd59f0d31683c03bdb690dffdf7f2a927223777f6b3e0912cc83f307: Status 404 returned error can't find the container with id a5eea323bd59f0d31683c03bdb690dffdf7f2a927223777f6b3e0912cc83f307
Oct 30 18:51:56.211692 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:56.211585    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"]
Oct 30 18:51:56.640463 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:56.640430    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr" event={"ID":"94081c24-324d-4214-9659-0e0628f7ad8d","Type":"ContainerStarted","Data":"75f18592ac3fac907ffb0dff7c93af95a56b50dfe7e3143d88cec578c7aed70f"}
Oct 30 18:51:56.643047 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:56.642975    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626" event={"ID":"a1632c13-8535-47c3-a331-c8dbc9141b3a","Type":"ContainerStarted","Data":"a5eea323bd59f0d31683c03bdb690dffdf7f2a927223777f6b3e0912cc83f307"}
Oct 30 18:51:56.644697 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:51:56.644588    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq" event={"ID":"0e42e57f-5a8b-4e1b-9db8-78367840f5f3","Type":"ContainerStarted","Data":"f979c66ac13599bc91b41629d8e8327a905d8b5109390364ef0fc244bb294056"}
Oct 30 18:52:06.143411 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.143372    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:52:06.143882 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.143473    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:52:06.143882 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.143520    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:52:06.724685 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.724647    5232 generic.go:356] "Generic (PLEG): container finished" podID="64796a8b-7b79-46f3-b84e-306f803430d9" containerID="81e0aaef08b58598baf512decc36b397950b7928a15950d86d2eba4ed7c853d6" exitCode=0
Oct 30 18:52:06.724882 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.724744    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl" event={"ID":"64796a8b-7b79-46f3-b84e-306f803430d9","Type":"ContainerDied","Data":"81e0aaef08b58598baf512decc36b397950b7928a15950d86d2eba4ed7c853d6"}
Oct 30 18:52:06.728561 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.728532    5232 generic.go:356] "Generic (PLEG): container finished" podID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerID="fe2d62fdfddc9253b5457b194aebee0c073c7cbc5e8b4ddf209974e25ef6d9ff" exitCode=0
Oct 30 18:52:06.728677 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.728635    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt" event={"ID":"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14","Type":"ContainerDied","Data":"fe2d62fdfddc9253b5457b194aebee0c073c7cbc5e8b4ddf209974e25ef6d9ff"}
Oct 30 18:52:06.733388 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.733363    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" event={"ID":"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a","Type":"ContainerStarted","Data":"a3a77426a8db554d2e408b6179ccfd8f6192b50b57099830c72fc99897e66acb"}
Oct 30 18:52:06.737381 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.737355    5232 generic.go:356] "Generic (PLEG): container finished" podID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerID="d03e91b7b70556f7c9ce8a946485bf99e2e066ebf5fa786ab4093367e1a37a96" exitCode=0
Oct 30 18:52:06.737480 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.737443    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n" event={"ID":"ca4d2130-307e-45f9-87b7-9621d7eb12d9","Type":"ContainerDied","Data":"d03e91b7b70556f7c9ce8a946485bf99e2e066ebf5fa786ab4093367e1a37a96"}
Oct 30 18:52:06.741520 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.741494    5232 generic.go:356] "Generic (PLEG): container finished" podID="f33f719b-232d-4393-92e4-282176c11a7d" containerID="8ab6545ea3db8ea0e2f8adf667f2e1a479c578b6c0b3b3a79df2fcbc71f63e81" exitCode=0
Oct 30 18:52:06.741612 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.741569    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph" event={"ID":"f33f719b-232d-4393-92e4-282176c11a7d","Type":"ContainerDied","Data":"8ab6545ea3db8ea0e2f8adf667f2e1a479c578b6c0b3b3a79df2fcbc71f63e81"}
Oct 30 18:52:06.746951 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.745679    5232 generic.go:356] "Generic (PLEG): container finished" podID="97d13881-9356-4a5b-8809-58262035eb5e" containerID="18b1bf515852be7fb3e5a161bbaba3ed191b0056c7c345411348bb57202760a6" exitCode=0
Oct 30 18:52:06.746951 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.745769    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n" event={"ID":"97d13881-9356-4a5b-8809-58262035eb5e","Type":"ContainerDied","Data":"18b1bf515852be7fb3e5a161bbaba3ed191b0056c7c345411348bb57202760a6"}
Oct 30 18:52:06.751500 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.750941    5232 generic.go:356] "Generic (PLEG): container finished" podID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerID="3a22ab9f368a613b0ef3b1fdeb42e35baa79b19762281843e4cf1d79f8d00ae9" exitCode=0
Oct 30 18:52:06.751500 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.750986    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp" event={"ID":"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7","Type":"ContainerDied","Data":"3a22ab9f368a613b0ef3b1fdeb42e35baa79b19762281843e4cf1d79f8d00ae9"}
Oct 30 18:52:06.946910 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:06.946826    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:52:07.002943 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.002091    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" podStartSLOduration=2.326990012 podStartE2EDuration="14.002070198s" totalImagesPullingTime="11.675080186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:07.001075505 +0000 UTC m=+319.585558503" watchObservedRunningTime="2026-10-30 18:52:07.002070198 +0000 UTC m=+319.586553279"
Oct 30 18:52:07.756114 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.756076    5232 generic.go:356] "Generic (PLEG): container finished" podID="94081c24-324d-4214-9659-0e0628f7ad8d" containerID="b2141356da8b562e81580146ade77c35d402f73f3a688bb0ace65e208dad6095" exitCode=0
Oct 30 18:52:07.756447 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.756169    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr" event={"ID":"94081c24-324d-4214-9659-0e0628f7ad8d","Type":"ContainerDied","Data":"b2141356da8b562e81580146ade77c35d402f73f3a688bb0ace65e208dad6095"}
Oct 30 18:52:07.757753 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.757730    5232 generic.go:356] "Generic (PLEG): container finished" podID="686bde4e-07e4-4381-9028-05db35bf4509" containerID="d8fee4839555ab75e52308c9eca7925cca990a15bbac5f8b4acf9e57368de80d" exitCode=0
Oct 30 18:52:07.757871 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.757802    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l" event={"ID":"686bde4e-07e4-4381-9028-05db35bf4509","Type":"ContainerDied","Data":"d8fee4839555ab75e52308c9eca7925cca990a15bbac5f8b4acf9e57368de80d"}
Oct 30 18:52:07.759325 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.759306    5232 generic.go:356] "Generic (PLEG): container finished" podID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerID="02f00e96d8c7510a70dba7b0a4cc8aa0a1a73118c050f41b1189199f566ed127" exitCode=0
Oct 30 18:52:07.759420 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.759369    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626" event={"ID":"a1632c13-8535-47c3-a331-c8dbc9141b3a","Type":"ContainerDied","Data":"02f00e96d8c7510a70dba7b0a4cc8aa0a1a73118c050f41b1189199f566ed127"}
Oct 30 18:52:07.760966 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.760947    5232 generic.go:356] "Generic (PLEG): container finished" podID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerID="3ca4801717081d4c29271d0f4a035cb1bb766fb0b78d8b1a5054555f35d4aac8" exitCode=0
Oct 30 18:52:07.761061 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.760973    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq" event={"ID":"0e42e57f-5a8b-4e1b-9db8-78367840f5f3","Type":"ContainerDied","Data":"3ca4801717081d4c29271d0f4a035cb1bb766fb0b78d8b1a5054555f35d4aac8"}
Oct 30 18:52:07.951554 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:07.951524    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:52:08.771755 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.771604    5232 generic.go:356] "Generic (PLEG): container finished" podID="97d13881-9356-4a5b-8809-58262035eb5e" containerID="738cac333125ba437e0a6d84d331630f239e7b314aef623e4167099d7ace30ad" exitCode=0
Oct 30 18:52:08.771755 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.771731    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n" event={"ID":"97d13881-9356-4a5b-8809-58262035eb5e","Type":"ContainerDied","Data":"738cac333125ba437e0a6d84d331630f239e7b314aef623e4167099d7ace30ad"}
Oct 30 18:52:08.774597 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.774497    5232 generic.go:356] "Generic (PLEG): container finished" podID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerID="6621a1903cff6469d5c6fc20283151c33b7c71f58a7bf4bd4d676bad1a6c06c2" exitCode=0
Oct 30 18:52:08.774772 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.774604    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp" event={"ID":"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7","Type":"ContainerDied","Data":"6621a1903cff6469d5c6fc20283151c33b7c71f58a7bf4bd4d676bad1a6c06c2"}
Oct 30 18:52:08.778208 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.778188    5232 generic.go:356] "Generic (PLEG): container finished" podID="64796a8b-7b79-46f3-b84e-306f803430d9" containerID="41acd5ae24b0c8ed63764398acc0e5ff743323854de30b72361b9aae4593a7e0" exitCode=0
Oct 30 18:52:08.778364 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.778325    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl" event={"ID":"64796a8b-7b79-46f3-b84e-306f803430d9","Type":"ContainerDied","Data":"41acd5ae24b0c8ed63764398acc0e5ff743323854de30b72361b9aae4593a7e0"}
Oct 30 18:52:08.780548 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.780511    5232 generic.go:356] "Generic (PLEG): container finished" podID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerID="c3d66806a404e0f574fe9237e63c0724a5fe0e3463a334d39879a7b59ea1c778" exitCode=0
Oct 30 18:52:08.781930 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.781911    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt" event={"ID":"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14","Type":"ContainerDied","Data":"c3d66806a404e0f574fe9237e63c0724a5fe0e3463a334d39879a7b59ea1c778"}
Oct 30 18:52:08.782014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.781956    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:52:08.783094 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:08.783077    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:52:09.792756 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:09.791668    5232 generic.go:356] "Generic (PLEG): container finished" podID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerID="da0cdfc52023a67de54366fb4352efc191d5d27c5590b37ab6467d2a20c34431" exitCode=0
Oct 30 18:52:09.792756 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:09.791878    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626" event={"ID":"a1632c13-8535-47c3-a331-c8dbc9141b3a","Type":"ContainerDied","Data":"da0cdfc52023a67de54366fb4352efc191d5d27c5590b37ab6467d2a20c34431"}
Oct 30 18:52:09.798005 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:09.797196    5232 generic.go:356] "Generic (PLEG): container finished" podID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerID="c7ef16cef6954029b7877c962921c9d3a453b898a5a4507cc9b632d7a01fc35a" exitCode=0
Oct 30 18:52:09.798005 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:09.797363    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq" event={"ID":"0e42e57f-5a8b-4e1b-9db8-78367840f5f3","Type":"ContainerDied","Data":"c7ef16cef6954029b7877c962921c9d3a453b898a5a4507cc9b632d7a01fc35a"}
Oct 30 18:52:09.805261 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:09.803993    5232 generic.go:356] "Generic (PLEG): container finished" podID="f33f719b-232d-4393-92e4-282176c11a7d" containerID="98dfe78852c1ef4cc5093f1c26548b01546f04f0be62bbe17db4af46d3484e7d" exitCode=0
Oct 30 18:52:09.805261 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:09.805175    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph" event={"ID":"f33f719b-232d-4393-92e4-282176c11a7d","Type":"ContainerDied","Data":"98dfe78852c1ef4cc5093f1c26548b01546f04f0be62bbe17db4af46d3484e7d"}
Oct 30 18:52:09.808642 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:09.808616    5232 generic.go:356] "Generic (PLEG): container finished" podID="94081c24-324d-4214-9659-0e0628f7ad8d" containerID="7520c9d6a3ef21c3a25f59c5291d9fc3c7b567495ab7827cdfb51766132bdc3d" exitCode=0
Oct 30 18:52:09.809553 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:09.809526    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr" event={"ID":"94081c24-324d-4214-9659-0e0628f7ad8d","Type":"ContainerDied","Data":"7520c9d6a3ef21c3a25f59c5291d9fc3c7b567495ab7827cdfb51766132bdc3d"}
Oct 30 18:52:10.821730 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:10.820164    5232 generic.go:356] "Generic (PLEG): container finished" podID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerID="30e47c4f3aa54b2a43b79d1acb0e3aaf855e03a35d51f5d395030a1e2b205400" exitCode=0
Oct 30 18:52:10.821730 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:10.821360    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n" event={"ID":"ca4d2130-307e-45f9-87b7-9621d7eb12d9","Type":"ContainerDied","Data":"30e47c4f3aa54b2a43b79d1acb0e3aaf855e03a35d51f5d395030a1e2b205400"}
Oct 30 18:52:14.866841 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:14.866739    5232 generic.go:356] "Generic (PLEG): container finished" podID="686bde4e-07e4-4381-9028-05db35bf4509" containerID="e46455a582a0d15f9d3411939b5910a4418cbff61694224abc14db73d8022567" exitCode=0
Oct 30 18:52:14.866841 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:14.866799    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l" event={"ID":"686bde4e-07e4-4381-9028-05db35bf4509","Type":"ContainerDied","Data":"e46455a582a0d15f9d3411939b5910a4418cbff61694224abc14db73d8022567"}
Oct 30 18:52:16.212856 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:52:16.212802    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:52:30.047183 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:30.047149    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp" event={"ID":"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7","Type":"ContainerStarted","Data":"c488568dd2fda65e17dc57d21671659a932c0a20af9ca64df9972dd3921792f8"}
Oct 30 18:52:30.052010 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:30.051965    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l" event={"ID":"686bde4e-07e4-4381-9028-05db35bf4509","Type":"ContainerStarted","Data":"9473f7d8188b0c0aeb0b3574c8f031fbe5c06623947cdd1dc7e616fed8b5b48c"}
Oct 30 18:52:30.183884 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:30.182210    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp" podStartSLOduration=2.991485602 podStartE2EDuration="43.182192417s" totalImagesPullingTime="40.190706815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:30.180874978 +0000 UTC m=+342.765357975" watchObservedRunningTime="2026-10-30 18:52:30.182192417 +0000 UTC m=+342.766675415"
Oct 30 18:52:30.297426 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:30.296269    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l" podStartSLOduration=2.616277594 podStartE2EDuration="41.296252603s" totalImagesPullingTime="38.679975009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:30.295599624 +0000 UTC m=+342.880082625" watchObservedRunningTime="2026-10-30 18:52:30.296252603 +0000 UTC m=+342.880735603"
Oct 30 18:52:31.064456 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.064414    5232 generic.go:356] "Generic (PLEG): container finished" podID="f33f719b-232d-4393-92e4-282176c11a7d" containerID="26d9552e7caea0e7e352cd26f24c78ca44cd9da967519fbae3432af74826a692" exitCode=0
Oct 30 18:52:31.064857 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.064495    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph" event={"ID":"f33f719b-232d-4393-92e4-282176c11a7d","Type":"ContainerDied","Data":"26d9552e7caea0e7e352cd26f24c78ca44cd9da967519fbae3432af74826a692"}
Oct 30 18:52:31.067293 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.067266    5232 generic.go:356] "Generic (PLEG): container finished" podID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerID="c488568dd2fda65e17dc57d21671659a932c0a20af9ca64df9972dd3921792f8" exitCode=0
Oct 30 18:52:31.067408 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.067357    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp" event={"ID":"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7","Type":"ContainerDied","Data":"c488568dd2fda65e17dc57d21671659a932c0a20af9ca64df9972dd3921792f8"}
Oct 30 18:52:31.069715 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.069685    5232 generic.go:356] "Generic (PLEG): container finished" podID="94081c24-324d-4214-9659-0e0628f7ad8d" containerID="bf4bea7f961bf2d17f59b8d9c363a939b26ee9a3b8e385f377971ec5b211d638" exitCode=0
Oct 30 18:52:31.069840 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.069736    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr" event={"ID":"94081c24-324d-4214-9659-0e0628f7ad8d","Type":"ContainerDied","Data":"bf4bea7f961bf2d17f59b8d9c363a939b26ee9a3b8e385f377971ec5b211d638"}
Oct 30 18:52:31.072273 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.072245    5232 generic.go:356] "Generic (PLEG): container finished" podID="64796a8b-7b79-46f3-b84e-306f803430d9" containerID="00a0f4cd9c4bb0f3b525ee25892458cafcba873bbc235dc6ca91ea33b898dc18" exitCode=0
Oct 30 18:52:31.072375 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.072326    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl" event={"ID":"64796a8b-7b79-46f3-b84e-306f803430d9","Type":"ContainerDied","Data":"00a0f4cd9c4bb0f3b525ee25892458cafcba873bbc235dc6ca91ea33b898dc18"}
Oct 30 18:52:31.074427 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.074403    5232 generic.go:356] "Generic (PLEG): container finished" podID="686bde4e-07e4-4381-9028-05db35bf4509" containerID="9473f7d8188b0c0aeb0b3574c8f031fbe5c06623947cdd1dc7e616fed8b5b48c" exitCode=0
Oct 30 18:52:31.074521 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.074491    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l" event={"ID":"686bde4e-07e4-4381-9028-05db35bf4509","Type":"ContainerDied","Data":"9473f7d8188b0c0aeb0b3574c8f031fbe5c06623947cdd1dc7e616fed8b5b48c"}
Oct 30 18:52:31.076895 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.076856    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626" event={"ID":"a1632c13-8535-47c3-a331-c8dbc9141b3a","Type":"ContainerStarted","Data":"74d46bded3f1ac870fcd393ecd58527d4180636a9c38a2d8a42e3c3335d44104"}
Oct 30 18:52:31.521387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:31.521324    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626" podStartSLOduration=3.365976362 podStartE2EDuration="36.521310524s" totalImagesPullingTime="33.155334162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:31.520508884 +0000 UTC m=+344.104991880" watchObservedRunningTime="2026-10-30 18:52:31.521310524 +0000 UTC m=+344.105793515"
Oct 30 18:52:32.082730 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.082652    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 18:52:32.083360 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.083339    5232 generic.go:356] "Generic (PLEG): container finished" podID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerID="1f4084ea5123270c9540bf1666c634fa57e86037aefd8cb7d971d0292fdf18ac" exitCode=1
Oct 30 18:52:32.083432 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.083416    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt" event={"ID":"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14","Type":"ContainerDied","Data":"1f4084ea5123270c9540bf1666c634fa57e86037aefd8cb7d971d0292fdf18ac"}
Oct 30 18:52:32.085485 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.085461    5232 generic.go:356] "Generic (PLEG): container finished" podID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerID="74d46bded3f1ac870fcd393ecd58527d4180636a9c38a2d8a42e3c3335d44104" exitCode=0
Oct 30 18:52:32.085575 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.085542    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626" event={"ID":"a1632c13-8535-47c3-a331-c8dbc9141b3a","Type":"ContainerDied","Data":"74d46bded3f1ac870fcd393ecd58527d4180636a9c38a2d8a42e3c3335d44104"}
Oct 30 18:52:32.087413 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.087396    5232 generic.go:356] "Generic (PLEG): container finished" podID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerID="4ff7a596f8592308ca6645db8bafdb744440c5f9daf5c38909e1a678ec111551" exitCode=0
Oct 30 18:52:32.087501 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.087459    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq" event={"ID":"0e42e57f-5a8b-4e1b-9db8-78367840f5f3","Type":"ContainerDied","Data":"4ff7a596f8592308ca6645db8bafdb744440c5f9daf5c38909e1a678ec111551"}
Oct 30 18:52:32.089466 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.089447    5232 generic.go:356] "Generic (PLEG): container finished" podID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerID="c7b4d4b12b37057ce29e6106bbe2de249d2a60531d5071e229b08976f0a771ae" exitCode=0
Oct 30 18:52:32.089561 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.089524    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n" event={"ID":"ca4d2130-307e-45f9-87b7-9621d7eb12d9","Type":"ContainerDied","Data":"c7b4d4b12b37057ce29e6106bbe2de249d2a60531d5071e229b08976f0a771ae"}
Oct 30 18:52:32.091431 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.091412    5232 generic.go:356] "Generic (PLEG): container finished" podID="97d13881-9356-4a5b-8809-58262035eb5e" containerID="a35e50621a6ccae6a6d34b45e8d852c1b22067c5bebd048ecd396d233e733824" exitCode=0
Oct 30 18:52:32.091530 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.091511    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n" event={"ID":"97d13881-9356-4a5b-8809-58262035eb5e","Type":"ContainerDied","Data":"a35e50621a6ccae6a6d34b45e8d852c1b22067c5bebd048ecd396d233e733824"}
Oct 30 18:52:32.245143 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.245120    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:52:32.285644 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.285620    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:52:32.289122 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.289099    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:52:32.308550 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.308526    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:52:32.338372 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.338349    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:52:32.401705 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401669    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-bundle\") pod \"64796a8b-7b79-46f3-b84e-306f803430d9\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") "
Oct 30 18:52:32.401873 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401725    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94081c24-324d-4214-9659-0e0628f7ad8d-kube-api-access-kkzp7\" (UniqueName: \"kubernetes.io/projected/94081c24-324d-4214-9659-0e0628f7ad8d-kube-api-access-kkzp7\") pod \"94081c24-324d-4214-9659-0e0628f7ad8d\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") "
Oct 30 18:52:32.401873 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401760    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-bundle\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-bundle\") pod \"686bde4e-07e4-4381-9028-05db35bf4509\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") "
Oct 30 18:52:32.401873 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401785    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-bundle\") pod \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") "
Oct 30 18:52:32.401873 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401836    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-kube-api-access-6dvvc\" (UniqueName: \"kubernetes.io/projected/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-kube-api-access-6dvvc\") pod \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") "
Oct 30 18:52:32.402079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401879    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64796a8b-7b79-46f3-b84e-306f803430d9-kube-api-access-xvcmm\" (UniqueName: \"kubernetes.io/projected/64796a8b-7b79-46f3-b84e-306f803430d9-kube-api-access-xvcmm\") pod \"64796a8b-7b79-46f3-b84e-306f803430d9\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") "
Oct 30 18:52:32.402079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401921    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/686bde4e-07e4-4381-9028-05db35bf4509-kube-api-access-kdrcx\" (UniqueName: \"kubernetes.io/projected/686bde4e-07e4-4381-9028-05db35bf4509-kube-api-access-kdrcx\") pod \"686bde4e-07e4-4381-9028-05db35bf4509\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") "
Oct 30 18:52:32.402079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401952    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-util\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-util\") pod \"94081c24-324d-4214-9659-0e0628f7ad8d\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") "
Oct 30 18:52:32.402079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.401983    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-util\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-util\") pod \"686bde4e-07e4-4381-9028-05db35bf4509\" (UID: \"686bde4e-07e4-4381-9028-05db35bf4509\") "
Oct 30 18:52:32.402079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.402039    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-util\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-util\") pod \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\" (UID: \"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7\") "
Oct 30 18:52:32.402079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.402061    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-util\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-util\") pod \"64796a8b-7b79-46f3-b84e-306f803430d9\" (UID: \"64796a8b-7b79-46f3-b84e-306f803430d9\") "
Oct 30 18:52:32.402386 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.402086    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-bundle\") pod \"94081c24-324d-4214-9659-0e0628f7ad8d\" (UID: \"94081c24-324d-4214-9659-0e0628f7ad8d\") "
Oct 30 18:52:32.402854 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.402801    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-bundle" pod "ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" (UID: "ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.403136 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.403107    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-bundle" pod "64796a8b-7b79-46f3-b84e-306f803430d9" (UID: "64796a8b-7b79-46f3-b84e-306f803430d9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.403223 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.403195    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-bundle" pod "686bde4e-07e4-4381-9028-05db35bf4509" (UID: "686bde4e-07e4-4381-9028-05db35bf4509"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.403276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.403202    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-bundle" pod "94081c24-324d-4214-9659-0e0628f7ad8d" (UID: "94081c24-324d-4214-9659-0e0628f7ad8d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.404587 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.404554    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94081c24-324d-4214-9659-0e0628f7ad8d-kube-api-access-kkzp7" pod "94081c24-324d-4214-9659-0e0628f7ad8d" (UID: "94081c24-324d-4214-9659-0e0628f7ad8d"). InnerVolumeSpecName "kube-api-access-kkzp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:32.404728 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.404636    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-kube-api-access-6dvvc" pod "ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" (UID: "ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7"). InnerVolumeSpecName "kube-api-access-6dvvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:32.404953 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.404927    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64796a8b-7b79-46f3-b84e-306f803430d9-kube-api-access-xvcmm" pod "64796a8b-7b79-46f3-b84e-306f803430d9" (UID: "64796a8b-7b79-46f3-b84e-306f803430d9"). InnerVolumeSpecName "kube-api-access-xvcmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:32.405064 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.405002    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/686bde4e-07e4-4381-9028-05db35bf4509-kube-api-access-kdrcx" pod "686bde4e-07e4-4381-9028-05db35bf4509" (UID: "686bde4e-07e4-4381-9028-05db35bf4509"). InnerVolumeSpecName "kube-api-access-kdrcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:32.409450 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.409429    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-util" pod "94081c24-324d-4214-9659-0e0628f7ad8d" (UID: "94081c24-324d-4214-9659-0e0628f7ad8d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.410170 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.410149    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-util" pod "64796a8b-7b79-46f3-b84e-306f803430d9" (UID: "64796a8b-7b79-46f3-b84e-306f803430d9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.411338 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.411315    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-util" pod "ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" (UID: "ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.411388 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.411313    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-util" pod "686bde4e-07e4-4381-9028-05db35bf4509" (UID: "686bde4e-07e4-4381-9028-05db35bf4509"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.503505 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503476    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f33f719b-232d-4393-92e4-282176c11a7d-kube-api-access-smbpd\" (UniqueName: \"kubernetes.io/projected/f33f719b-232d-4393-92e4-282176c11a7d-kube-api-access-smbpd\") pod \"f33f719b-232d-4393-92e4-282176c11a7d\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") "
Oct 30 18:52:32.503690 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503531    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-util\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-util\") pod \"f33f719b-232d-4393-92e4-282176c11a7d\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") "
Oct 30 18:52:32.503690 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503556    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-bundle\") pod \"f33f719b-232d-4393-92e4-282176c11a7d\" (UID: \"f33f719b-232d-4393-92e4-282176c11a7d\") "
Oct 30 18:52:32.503786 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503769    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503794    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dvvc\" (UniqueName: \"kubernetes.io/projected/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-kube-api-access-6dvvc\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503830    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvcmm\" (UniqueName: \"kubernetes.io/projected/64796a8b-7b79-46f3-b84e-306f803430d9-kube-api-access-xvcmm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503845    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdrcx\" (UniqueName: \"kubernetes.io/projected/686bde4e-07e4-4381-9028-05db35bf4509-kube-api-access-kdrcx\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503857    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503870    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503885    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503897    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503913    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94081c24-324d-4214-9659-0e0628f7ad8d-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503925    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64796a8b-7b79-46f3-b84e-306f803430d9-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503938    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkzp7\" (UniqueName: \"kubernetes.io/projected/94081c24-324d-4214-9659-0e0628f7ad8d-kube-api-access-kkzp7\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.503957 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.503953    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686bde4e-07e4-4381-9028-05db35bf4509-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.504218 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.504120    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-bundle" pod "f33f719b-232d-4393-92e4-282176c11a7d" (UID: "f33f719b-232d-4393-92e4-282176c11a7d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.505637 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.505610    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f33f719b-232d-4393-92e4-282176c11a7d-kube-api-access-smbpd" pod "f33f719b-232d-4393-92e4-282176c11a7d" (UID: "f33f719b-232d-4393-92e4-282176c11a7d"). InnerVolumeSpecName "kube-api-access-smbpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:32.509028 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.509009    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-util" pod "f33f719b-232d-4393-92e4-282176c11a7d" (UID: "f33f719b-232d-4393-92e4-282176c11a7d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:32.604379 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.604293    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smbpd\" (UniqueName: \"kubernetes.io/projected/f33f719b-232d-4393-92e4-282176c11a7d-kube-api-access-smbpd\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.604379 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.604324    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:32.604379 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:32.604337    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f33f719b-232d-4393-92e4-282176c11a7d-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.097088 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.097064    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph"
Oct 30 18:52:33.097482 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.097056    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78b5ph" event={"ID":"f33f719b-232d-4393-92e4-282176c11a7d","Type":"ContainerDied","Data":"4493501c21b33b2a6411fe9010d3fc9322a207b83604d84851e91fc8580dd7b8"}
Oct 30 18:52:33.097482 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.097188    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4493501c21b33b2a6411fe9010d3fc9322a207b83604d84851e91fc8580dd7b8"
Oct 30 18:52:33.098955 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.098936    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp"
Oct 30 18:52:33.099081 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.098934    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cfszfp" event={"ID":"ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7","Type":"ContainerDied","Data":"e98dab2b764ddba946fbec4deedbaa81b05bfdc1a66c2d8442f4e5644e7d580f"}
Oct 30 18:52:33.099081 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.099056    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e98dab2b764ddba946fbec4deedbaa81b05bfdc1a66c2d8442f4e5644e7d580f"
Oct 30 18:52:33.100861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.100835    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr" event={"ID":"94081c24-324d-4214-9659-0e0628f7ad8d","Type":"ContainerDied","Data":"75f18592ac3fac907ffb0dff7c93af95a56b50dfe7e3143d88cec578c7aed70f"}
Oct 30 18:52:33.100973 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.100882    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75f18592ac3fac907ffb0dff7c93af95a56b50dfe7e3143d88cec578c7aed70f"
Oct 30 18:52:33.100973 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.100849    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xwxxr"
Oct 30 18:52:33.102874 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.102856    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl"
Oct 30 18:52:33.102985 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.102886    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqr9pl" event={"ID":"64796a8b-7b79-46f3-b84e-306f803430d9","Type":"ContainerDied","Data":"401247afc65dab623f085bdb1346af680b52224d735742a3c061e485fedbd98f"}
Oct 30 18:52:33.102985 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.102920    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="401247afc65dab623f085bdb1346af680b52224d735742a3c061e485fedbd98f"
Oct 30 18:52:33.105150 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.105108    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l"
Oct 30 18:52:33.105150 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.105140    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445q94l" event={"ID":"686bde4e-07e4-4381-9028-05db35bf4509","Type":"ContainerDied","Data":"e7c9b59f701ae6ece73fe541442d8b0603d2a1095ea02e201ee2c83b7da5642b"}
Oct 30 18:52:33.105311 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.105177    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7c9b59f701ae6ece73fe541442d8b0603d2a1095ea02e201ee2c83b7da5642b"
Oct 30 18:52:33.271310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.271284    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:52:33.324271 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.324249    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:52:33.329476 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.329453    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:52:33.332585 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.332564    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:52:33.333737 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.333722    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 18:52:33.335505 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.335486    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:52:33.410896 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.410784    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca4d2130-307e-45f9-87b7-9621d7eb12d9-kube-api-access-rjv2m\" (UniqueName: \"kubernetes.io/projected/ca4d2130-307e-45f9-87b7-9621d7eb12d9-kube-api-access-rjv2m\") pod \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") "
Oct 30 18:52:33.410896 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.410882    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-bundle\") pod \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") "
Oct 30 18:52:33.411121 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.410930    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-util\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-util\") pod \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\" (UID: \"ca4d2130-307e-45f9-87b7-9621d7eb12d9\") "
Oct 30 18:52:33.411477 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.411453    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-bundle" pod "ca4d2130-307e-45f9-87b7-9621d7eb12d9" (UID: "ca4d2130-307e-45f9-87b7-9621d7eb12d9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.412892 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.412861    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca4d2130-307e-45f9-87b7-9621d7eb12d9-kube-api-access-rjv2m" pod "ca4d2130-307e-45f9-87b7-9621d7eb12d9" (UID: "ca4d2130-307e-45f9-87b7-9621d7eb12d9"). InnerVolumeSpecName "kube-api-access-rjv2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.416051 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.416031    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-util" pod "ca4d2130-307e-45f9-87b7-9621d7eb12d9" (UID: "ca4d2130-307e-45f9-87b7-9621d7eb12d9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.511974 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.511937    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-util\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-util\") pod \"a1632c13-8535-47c3-a331-c8dbc9141b3a\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") "
Oct 30 18:52:33.511974 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.511979    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-bundle\") pod \"97d13881-9356-4a5b-8809-58262035eb5e\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") "
Oct 30 18:52:33.512212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512008    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-kube-api-access-j95sc\" (UniqueName: \"kubernetes.io/projected/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-kube-api-access-j95sc\") pod \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") "
Oct 30 18:52:33.512212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512038    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-util\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-util\") pod \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") "
Oct 30 18:52:33.512212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512055    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-util\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-util\") pod \"97d13881-9356-4a5b-8809-58262035eb5e\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") "
Oct 30 18:52:33.512212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512073    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-bundle\") pod \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") "
Oct 30 18:52:33.512212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512133    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-bundle\") pod \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") "
Oct 30 18:52:33.512212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512178    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-bundle\") pod \"a1632c13-8535-47c3-a331-c8dbc9141b3a\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") "
Oct 30 18:52:33.512212 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512207    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97d13881-9356-4a5b-8809-58262035eb5e-kube-api-access-x9gpv\" (UniqueName: \"kubernetes.io/projected/97d13881-9356-4a5b-8809-58262035eb5e-kube-api-access-x9gpv\") pod \"97d13881-9356-4a5b-8809-58262035eb5e\" (UID: \"97d13881-9356-4a5b-8809-58262035eb5e\") "
Oct 30 18:52:33.512551 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512245    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1632c13-8535-47c3-a331-c8dbc9141b3a-kube-api-access-p6hmj\" (UniqueName: \"kubernetes.io/projected/a1632c13-8535-47c3-a331-c8dbc9141b3a-kube-api-access-p6hmj\") pod \"a1632c13-8535-47c3-a331-c8dbc9141b3a\" (UID: \"a1632c13-8535-47c3-a331-c8dbc9141b3a\") "
Oct 30 18:52:33.512551 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512277    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-util\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-util\") pod \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\" (UID: \"0e42e57f-5a8b-4e1b-9db8-78367840f5f3\") "
Oct 30 18:52:33.512551 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512307    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-kube-api-access-wnn48\" (UniqueName: \"kubernetes.io/projected/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-kube-api-access-wnn48\") pod \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\" (UID: \"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14\") "
Oct 30 18:52:33.512551 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512519    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.512551 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512542    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4d2130-307e-45f9-87b7-9621d7eb12d9-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.512796 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512555    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjv2m\" (UniqueName: \"kubernetes.io/projected/ca4d2130-307e-45f9-87b7-9621d7eb12d9-kube-api-access-rjv2m\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.512796 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512579    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-bundle" pod "0e42e57f-5a8b-4e1b-9db8-78367840f5f3" (UID: "0e42e57f-5a8b-4e1b-9db8-78367840f5f3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.513076 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.512717    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-bundle" pod "97d13881-9356-4a5b-8809-58262035eb5e" (UID: "97d13881-9356-4a5b-8809-58262035eb5e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.513497 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.513460    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-bundle" pod "0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" (UID: "0b8e28d4-3c32-4801-8fa8-4a19a59fdb14"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.513653 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.513632    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-bundle" pod "a1632c13-8535-47c3-a331-c8dbc9141b3a" (UID: "a1632c13-8535-47c3-a331-c8dbc9141b3a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.515108 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.515055    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-kube-api-access-j95sc" pod "0e42e57f-5a8b-4e1b-9db8-78367840f5f3" (UID: "0e42e57f-5a8b-4e1b-9db8-78367840f5f3"). InnerVolumeSpecName "kube-api-access-j95sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.515108 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.515079    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-kube-api-access-wnn48" pod "0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" (UID: "0b8e28d4-3c32-4801-8fa8-4a19a59fdb14"). InnerVolumeSpecName "kube-api-access-wnn48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.516468 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.516444    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97d13881-9356-4a5b-8809-58262035eb5e-kube-api-access-x9gpv" pod "97d13881-9356-4a5b-8809-58262035eb5e" (UID: "97d13881-9356-4a5b-8809-58262035eb5e"). InnerVolumeSpecName "kube-api-access-x9gpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.516750 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.516731    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1632c13-8535-47c3-a331-c8dbc9141b3a-kube-api-access-p6hmj" pod "a1632c13-8535-47c3-a331-c8dbc9141b3a" (UID: "a1632c13-8535-47c3-a331-c8dbc9141b3a"). InnerVolumeSpecName "kube-api-access-p6hmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.519877 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.519855    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-util" pod "0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" (UID: "0b8e28d4-3c32-4801-8fa8-4a19a59fdb14"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.519983 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.519965    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-util" pod "97d13881-9356-4a5b-8809-58262035eb5e" (UID: "97d13881-9356-4a5b-8809-58262035eb5e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.520945 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.520920    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-util" pod "a1632c13-8535-47c3-a331-c8dbc9141b3a" (UID: "a1632c13-8535-47c3-a331-c8dbc9141b3a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.521333 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.521316    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-util" pod "0e42e57f-5a8b-4e1b-9db8-78367840f5f3" (UID: "0e42e57f-5a8b-4e1b-9db8-78367840f5f3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.613125 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613093    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6hmj\" (UniqueName: \"kubernetes.io/projected/a1632c13-8535-47c3-a331-c8dbc9141b3a-kube-api-access-p6hmj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613125 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613121    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613125 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613132    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnn48\" (UniqueName: \"kubernetes.io/projected/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-kube-api-access-wnn48\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613141    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613150    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613159    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j95sc\" (UniqueName: \"kubernetes.io/projected/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-kube-api-access-j95sc\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613167    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613175    5232 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97d13881-9356-4a5b-8809-58262035eb5e-util\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613182    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e42e57f-5a8b-4e1b-9db8-78367840f5f3-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613190    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8e28d4-3c32-4801-8fa8-4a19a59fdb14-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613197    5232 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1632c13-8535-47c3-a331-c8dbc9141b3a-bundle\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.613344 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:33.613204    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9gpv\" (UniqueName: \"kubernetes.io/projected/97d13881-9356-4a5b-8809-58262035eb5e-kube-api-access-x9gpv\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:34.111366 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.111339    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n"
Oct 30 18:52:34.111836 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.111329    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7147s2n" event={"ID":"97d13881-9356-4a5b-8809-58262035eb5e","Type":"ContainerDied","Data":"abb22a252edef9c7bb72b2e220cb6b4c77625522192e36ec73457d4d7e14b9af"}
Oct 30 18:52:34.111836 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.111458    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abb22a252edef9c7bb72b2e220cb6b4c77625522192e36ec73457d4d7e14b9af"
Oct 30 18:52:34.112677 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.112657    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 18:52:34.113466 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.113440    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt"
Oct 30 18:52:34.113567 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.113443    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt" event={"ID":"0b8e28d4-3c32-4801-8fa8-4a19a59fdb14","Type":"ContainerDied","Data":"4cc8d3e9154e0fa00f39808c341e55a703f030412be9ab524bee7e8dee3a2e4f"}
Oct 30 18:52:34.113626 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.113587    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cc8d3e9154e0fa00f39808c341e55a703f030412be9ab524bee7e8dee3a2e4f"
Oct 30 18:52:34.115732 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.115709    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626" event={"ID":"a1632c13-8535-47c3-a331-c8dbc9141b3a","Type":"ContainerDied","Data":"a5eea323bd59f0d31683c03bdb690dffdf7f2a927223777f6b3e0912cc83f307"}
Oct 30 18:52:34.115844 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.115748    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5eea323bd59f0d31683c03bdb690dffdf7f2a927223777f6b3e0912cc83f307"
Oct 30 18:52:34.115914 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.115858    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxn626"
Oct 30 18:52:34.117767 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.117609    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq" event={"ID":"0e42e57f-5a8b-4e1b-9db8-78367840f5f3","Type":"ContainerDied","Data":"f979c66ac13599bc91b41629d8e8327a905d8b5109390364ef0fc244bb294056"}
Oct 30 18:52:34.117767 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.117645    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f979c66ac13599bc91b41629d8e8327a905d8b5109390364ef0fc244bb294056"
Oct 30 18:52:34.117767 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.117612    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gc2jq"
Oct 30 18:52:34.119723 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.119690    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n" event={"ID":"ca4d2130-307e-45f9-87b7-9621d7eb12d9","Type":"ContainerDied","Data":"5747392234b94bfe16d165fe5cdd19a5e8adb255f9ff450550a90313ec89d6c8"}
Oct 30 18:52:34.119874 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.119726    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5747392234b94bfe16d165fe5cdd19a5e8adb255f9ff450550a90313ec89d6c8"
Oct 30 18:52:34.119874 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:34.119746    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651clnm5n"
Oct 30 18:52:37.046188 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046156    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"]
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046424    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046434    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046445    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046450    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046457    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046461    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046468    5232 cpu_manager.go:414] "containerMap: removing container" podUID="64796a8b-7b79-46f3-b84e-306f803430d9" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046473    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64796a8b-7b79-46f3-b84e-306f803430d9" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046484    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046488    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046496    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f33f719b-232d-4393-92e4-282176c11a7d" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046500    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f33f719b-232d-4393-92e4-282176c11a7d" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046506    5232 cpu_manager.go:414] "containerMap: removing container" podUID="97d13881-9356-4a5b-8809-58262035eb5e" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046511    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97d13881-9356-4a5b-8809-58262035eb5e" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046517    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046521    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046527    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046531    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046538    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046543    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046549    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046552    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046558    5232 cpu_manager.go:414] "containerMap: removing container" podUID="97d13881-9356-4a5b-8809-58262035eb5e" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046562    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97d13881-9356-4a5b-8809-58262035eb5e" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046567    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046571    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046579    5232 cpu_manager.go:414] "containerMap: removing container" podUID="64796a8b-7b79-46f3-b84e-306f803430d9" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046583    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64796a8b-7b79-46f3-b84e-306f803430d9" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046588    5232 cpu_manager.go:414] "containerMap: removing container" podUID="97d13881-9356-4a5b-8809-58262035eb5e" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046592    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97d13881-9356-4a5b-8809-58262035eb5e" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046597    5232 cpu_manager.go:414] "containerMap: removing container" podUID="94081c24-324d-4214-9659-0e0628f7ad8d" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046602    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94081c24-324d-4214-9659-0e0628f7ad8d" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046607    5232 cpu_manager.go:414] "containerMap: removing container" podUID="94081c24-324d-4214-9659-0e0628f7ad8d" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046613    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94081c24-324d-4214-9659-0e0628f7ad8d" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046621    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046626    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerName="util"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046632    5232 cpu_manager.go:414] "containerMap: removing container" podUID="64796a8b-7b79-46f3-b84e-306f803430d9" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046636    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64796a8b-7b79-46f3-b84e-306f803430d9" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046641    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f33f719b-232d-4393-92e4-282176c11a7d" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046645    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f33f719b-232d-4393-92e4-282176c11a7d" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046652    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046658    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046670    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046677    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerName="pull"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046684    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f33f719b-232d-4393-92e4-282176c11a7d" containerName="extract"
Oct 30 18:52:37.046670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046689    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f33f719b-232d-4393-92e4-282176c11a7d" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046695    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046700    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046708    5232 cpu_manager.go:414] "containerMap: removing container" podUID="686bde4e-07e4-4381-9028-05db35bf4509" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046712    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="686bde4e-07e4-4381-9028-05db35bf4509" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046720    5232 cpu_manager.go:414] "containerMap: removing container" podUID="94081c24-324d-4214-9659-0e0628f7ad8d" containerName="util"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046724    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94081c24-324d-4214-9659-0e0628f7ad8d" containerName="util"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046730    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerName="pull"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046734    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerName="pull"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046739    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046743    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046750    5232 cpu_manager.go:414] "containerMap: removing container" podUID="686bde4e-07e4-4381-9028-05db35bf4509" containerName="util"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046754    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="686bde4e-07e4-4381-9028-05db35bf4509" containerName="util"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046759    5232 cpu_manager.go:414] "containerMap: removing container" podUID="686bde4e-07e4-4381-9028-05db35bf4509" containerName="pull"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046763    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="686bde4e-07e4-4381-9028-05db35bf4509" containerName="pull"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046799    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="64796a8b-7b79-46f3-b84e-306f803430d9" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046806    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca4d2130-307e-45f9-87b7-9621d7eb12d9" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046825    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="97d13881-9356-4a5b-8809-58262035eb5e" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046831    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b8e28d4-3c32-4801-8fa8-4a19a59fdb14" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046837    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="94081c24-324d-4214-9659-0e0628f7ad8d" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046842    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca19d4b3-5ea8-412e-b0d2-a0fc3b3f01a7" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046846    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="f33f719b-232d-4393-92e4-282176c11a7d" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046851    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1632c13-8535-47c3-a331-c8dbc9141b3a" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046856    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="686bde4e-07e4-4381-9028-05db35bf4509" containerName="extract"
Oct 30 18:52:37.048014 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.046861    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e42e57f-5a8b-4e1b-9db8-78367840f5f3" containerName="extract"
Oct 30 18:52:37.049907 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.049891    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.105430 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.105395    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 18:52:37.105597 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.105462    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 18:52:37.105597 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.105544    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 18:52:37.152418 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.152388    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 18:52:37.152586 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.152433    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-wxrjr\""
Oct 30 18:52:37.152586 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.152471    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 18:52:37.161933 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.161912    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"]
Oct 30 18:52:37.241102 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.241071    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whpgn\" (UniqueName: \"kubernetes.io/projected/4d3bbf75-6468-41cd-8a5e-413659d4b475-kube-api-access-whpgn\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.241250 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.241119    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-metrics-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.241250 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.241141    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d3bbf75-6468-41cd-8a5e-413659d4b475-tmp\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.241250 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.241220    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d3bbf75-6468-41cd-8a5e-413659d4b475-manifests\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.241357 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.241251    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-webhook-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.241357 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.241283    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.342046 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.342018    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whpgn\" (UniqueName: \"kubernetes.io/projected/4d3bbf75-6468-41cd-8a5e-413659d4b475-kube-api-access-whpgn\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.342213 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.342062    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-metrics-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.342213 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.342084    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d3bbf75-6468-41cd-8a5e-413659d4b475-tmp\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.342213 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.342104    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d3bbf75-6468-41cd-8a5e-413659d4b475-manifests\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.342213 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.342131    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-webhook-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.342413 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.342245    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.342600 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.342577    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d3bbf75-6468-41cd-8a5e-413659d4b475-manifests\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.344601 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.344576    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-webhook-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.344706 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.344605    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.344768 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.344708    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d3bbf75-6468-41cd-8a5e-413659d4b475-tmp\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.344868 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.344850    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d3bbf75-6468-41cd-8a5e-413659d4b475-metrics-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.401491 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.401459    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whpgn\" (UniqueName: \"kubernetes.io/projected/4d3bbf75-6468-41cd-8a5e-413659d4b475-kube-api-access-whpgn\") pod \"opendatahub-operator-controller-manager-575fc98dd-bjnvh\" (UID: \"4d3bbf75-6468-41cd-8a5e-413659d4b475\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:37.664004 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:37.663916    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:38.230355 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:52:38.230329    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3bbf75_6468_41cd_8a5e_413659d4b475.slice/crio-73433293020f0e8ea5128437c951573f729dc106b6f4d419c76cf26a39c5ef27 WatchSource:0}: Error finding container 73433293020f0e8ea5128437c951573f729dc106b6f4d419c76cf26a39c5ef27: Status 404 returned error can't find the container with id 73433293020f0e8ea5128437c951573f729dc106b6f4d419c76cf26a39c5ef27
Oct 30 18:52:38.251145 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:38.251116    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"]
Oct 30 18:52:39.147330 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:39.147288    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh" event={"ID":"4d3bbf75-6468-41cd-8a5e-413659d4b475","Type":"ContainerStarted","Data":"73433293020f0e8ea5128437c951573f729dc106b6f4d419c76cf26a39c5ef27"}
Oct 30 18:52:42.161354 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:42.161323    5232 generic.go:356] "Generic (PLEG): container finished" podID="4d3bbf75-6468-41cd-8a5e-413659d4b475" containerID="7207d8ec7155f493a473171dabe5b9fa9ed6a25f6e9572c9983017f132f346ef" exitCode=0
Oct 30 18:52:42.161687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:42.161408    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh" event={"ID":"4d3bbf75-6468-41cd-8a5e-413659d4b475","Type":"ContainerDied","Data":"7207d8ec7155f493a473171dabe5b9fa9ed6a25f6e9572c9983017f132f346ef"}
Oct 30 18:52:43.167536 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:43.167503    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh" event={"ID":"4d3bbf75-6468-41cd-8a5e-413659d4b475","Type":"ContainerStarted","Data":"301fa918c68ac3bac6381d87c343101f48aebea892726818f6f1f0322146c30b"}
Oct 30 18:52:43.167979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:43.167651    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:43.404319 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:43.404262    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh" podStartSLOduration=2.739304288 podStartE2EDuration="7.404247948s" totalImagesPullingTime="4.226391519s" totalInitContainerRuntime="438.552141ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:43.357080014 +0000 UTC m=+355.941563024" watchObservedRunningTime="2026-10-30 18:52:43.404247948 +0000 UTC m=+355.988730944"
Oct 30 18:52:45.547095 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.547064    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"]
Oct 30 18:52:45.549525 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.549506    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"
Oct 30 18:52:45.605125 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.605078    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgs7q\" (UniqueName: \"kubernetes.io/projected/a6d946b1-cb85-4bf5-8144-3a121af0b5d2-kube-api-access-jgs7q\") pod \"authorino-operator-7b7bb4bc77-jd8zk\" (UID: \"a6d946b1-cb85-4bf5-8144-3a121af0b5d2\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"
Oct 30 18:52:45.605313 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.605162    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 18:52:45.652605 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.652568    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 18:52:45.652605 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.652571    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-p2kqg\""
Oct 30 18:52:45.654064 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.654044    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"]
Oct 30 18:52:45.706023 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.705987    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgs7q\" (UniqueName: \"kubernetes.io/projected/a6d946b1-cb85-4bf5-8144-3a121af0b5d2-kube-api-access-jgs7q\") pod \"authorino-operator-7b7bb4bc77-jd8zk\" (UID: \"a6d946b1-cb85-4bf5-8144-3a121af0b5d2\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"
Oct 30 18:52:45.767986 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.767949    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgs7q\" (UniqueName: \"kubernetes.io/projected/a6d946b1-cb85-4bf5-8144-3a121af0b5d2-kube-api-access-jgs7q\") pod \"authorino-operator-7b7bb4bc77-jd8zk\" (UID: \"a6d946b1-cb85-4bf5-8144-3a121af0b5d2\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"
Oct 30 18:52:45.860349 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:45.860319    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"
Oct 30 18:52:46.186133 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:52:46.186043    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:52:46.226759 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:46.226723    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"]
Oct 30 18:52:46.234056 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:52:46.234028    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d946b1_cb85_4bf5_8144_3a121af0b5d2.slice/crio-0d4d08ff17d7d9889dc4f670d28019b06db21a5e288e75be95781ebdc8a162bf WatchSource:0}: Error finding container 0d4d08ff17d7d9889dc4f670d28019b06db21a5e288e75be95781ebdc8a162bf: Status 404 returned error can't find the container with id 0d4d08ff17d7d9889dc4f670d28019b06db21a5e288e75be95781ebdc8a162bf
Oct 30 18:52:47.118443 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.118414    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"]
Oct 30 18:52:47.121501 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.121479    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"
Oct 30 18:52:47.176436 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.176408    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:52:47.176637 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.176513    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 30 18:52:47.187243 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.187210    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk" event={"ID":"a6d946b1-cb85-4bf5-8144-3a121af0b5d2","Type":"ContainerStarted","Data":"0d4d08ff17d7d9889dc4f670d28019b06db21a5e288e75be95781ebdc8a162bf"}
Oct 30 18:52:47.217190 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.217162    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9hgn\" (UniqueName: \"kubernetes.io/projected/da989372-b5b7-4999-900c-de5d35b93abc-kube-api-access-j9hgn\") pod \"jobset-operator-674cd59dc5-vkvfd\" (UID: \"da989372-b5b7-4999-900c-de5d35b93abc\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"
Oct 30 18:52:47.217336 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.217202    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da989372-b5b7-4999-900c-de5d35b93abc-tmp\") pod \"jobset-operator-674cd59dc5-vkvfd\" (UID: \"da989372-b5b7-4999-900c-de5d35b93abc\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"
Oct 30 18:52:47.224075 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.224053    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-267sm\""
Oct 30 18:52:47.225443 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.225408    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"]
Oct 30 18:52:47.318304 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.318269    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9hgn\" (UniqueName: \"kubernetes.io/projected/da989372-b5b7-4999-900c-de5d35b93abc-kube-api-access-j9hgn\") pod \"jobset-operator-674cd59dc5-vkvfd\" (UID: \"da989372-b5b7-4999-900c-de5d35b93abc\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"
Oct 30 18:52:47.318469 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.318313    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da989372-b5b7-4999-900c-de5d35b93abc-tmp\") pod \"jobset-operator-674cd59dc5-vkvfd\" (UID: \"da989372-b5b7-4999-900c-de5d35b93abc\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"
Oct 30 18:52:47.318659 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.318643    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da989372-b5b7-4999-900c-de5d35b93abc-tmp\") pod \"jobset-operator-674cd59dc5-vkvfd\" (UID: \"da989372-b5b7-4999-900c-de5d35b93abc\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"
Oct 30 18:52:47.377517 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.377435    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9hgn\" (UniqueName: \"kubernetes.io/projected/da989372-b5b7-4999-900c-de5d35b93abc-kube-api-access-j9hgn\") pod \"jobset-operator-674cd59dc5-vkvfd\" (UID: \"da989372-b5b7-4999-900c-de5d35b93abc\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"
Oct 30 18:52:47.451614 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.451570    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"
Oct 30 18:52:47.824559 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:47.824471    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd"]
Oct 30 18:52:47.832009 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:52:47.831976    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda989372_b5b7_4999_900c_de5d35b93abc.slice/crio-b114bf609123313592a3e39b395a1c2618061d678efe4baec1e3f484d3d72af0 WatchSource:0}: Error finding container b114bf609123313592a3e39b395a1c2618061d678efe4baec1e3f484d3d72af0: Status 404 returned error can't find the container with id b114bf609123313592a3e39b395a1c2618061d678efe4baec1e3f484d3d72af0
Oct 30 18:52:48.194916 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:48.194876    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd" event={"ID":"da989372-b5b7-4999-900c-de5d35b93abc","Type":"ContainerStarted","Data":"b114bf609123313592a3e39b395a1c2618061d678efe4baec1e3f484d3d72af0"}
Oct 30 18:52:50.205311 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:50.205274    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk" event={"ID":"a6d946b1-cb85-4bf5-8144-3a121af0b5d2","Type":"ContainerStarted","Data":"cd4a89c43890f4e06f24f9d9f097ea8c38fa7449bb4264f49a9e752b042e4b24"}
Oct 30 18:52:50.205684 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:50.205357    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"
Oct 30 18:52:50.326995 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:50.326935    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk" podStartSLOduration=1.796184877 podStartE2EDuration="5.326921774s" totalImagesPullingTime="3.530736897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:50.326155658 +0000 UTC m=+362.910638655" watchObservedRunningTime="2026-10-30 18:52:50.326921774 +0000 UTC m=+362.911404772"
Oct 30 18:52:51.211768 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:51.211734    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd" event={"ID":"da989372-b5b7-4999-900c-de5d35b93abc","Type":"ContainerStarted","Data":"7c981ba622cb3053be4ed035979f495b257ef954494a2e439ea2117114837b8e"}
Oct 30 18:52:51.360792 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:51.360685    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-vkvfd" podStartSLOduration=1.102709519 podStartE2EDuration="4.360669211s" totalImagesPullingTime="3.257959692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:51.359368571 +0000 UTC m=+363.943851568" watchObservedRunningTime="2026-10-30 18:52:51.360669211 +0000 UTC m=+363.945152202"
Oct 30 18:52:54.024169 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.024137    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"]
Oct 30 18:52:54.026531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.026515    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:52:54.064503 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.064469    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-t7tkj\" (UID: \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:52:54.064637 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.064579    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgwk4\" (UniqueName: \"kubernetes.io/projected/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-kube-api-access-xgwk4\") pod \"kuadrant-operator-controller-manager-cf8469d7b-t7tkj\" (UID: \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:52:54.080461 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.080439    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t5xns\""
Oct 30 18:52:54.129469 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.129439    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"]
Oct 30 18:52:54.165099 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.165068    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgwk4\" (UniqueName: \"kubernetes.io/projected/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-kube-api-access-xgwk4\") pod \"kuadrant-operator-controller-manager-cf8469d7b-t7tkj\" (UID: \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:52:54.165246 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.165109    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-t7tkj\" (UID: \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:52:54.165492 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.165472    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-t7tkj\" (UID: \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:52:54.174866 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.174844    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-bjnvh"
Oct 30 18:52:54.225155 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.225126    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgwk4\" (UniqueName: \"kubernetes.io/projected/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-kube-api-access-xgwk4\") pod \"kuadrant-operator-controller-manager-cf8469d7b-t7tkj\" (UID: \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:52:54.337975 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.337935    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:52:54.761903 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:54.761832    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"]
Oct 30 18:52:54.767774 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:52:54.767750    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5726fe0_afd3_4abf_a89e_9b9ea85bbe4a.slice/crio-52b064c39a6b489b17dea41a9d40b784a1806585df91272fa7c302a16deb1a43 WatchSource:0}: Error finding container 52b064c39a6b489b17dea41a9d40b784a1806585df91272fa7c302a16deb1a43: Status 404 returned error can't find the container with id 52b064c39a6b489b17dea41a9d40b784a1806585df91272fa7c302a16deb1a43
Oct 30 18:52:55.231112 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:55.231078    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj" event={"ID":"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a","Type":"ContainerStarted","Data":"52b064c39a6b489b17dea41a9d40b784a1806585df91272fa7c302a16deb1a43"}
Oct 30 18:52:58.316162 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.316087    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"]
Oct 30 18:52:58.328148 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.328123    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"
Oct 30 18:52:58.373607 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.373571    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"]
Oct 30 18:52:58.395309 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.395275    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/359a6ecd-500d-4916-a2bb-dc8d2563cdb5-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6\" (UID: \"359a6ecd-500d-4916-a2bb-dc8d2563cdb5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"
Oct 30 18:52:58.395483 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.395331    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dvx8\" (UniqueName: \"kubernetes.io/projected/359a6ecd-500d-4916-a2bb-dc8d2563cdb5-kube-api-access-4dvx8\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6\" (UID: \"359a6ecd-500d-4916-a2bb-dc8d2563cdb5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"
Oct 30 18:52:58.419610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.419580    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:52:58.419610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.419601    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-m5kr8\""
Oct 30 18:52:58.419851 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.419687    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 18:52:58.495888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.495838    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dvx8\" (UniqueName: \"kubernetes.io/projected/359a6ecd-500d-4916-a2bb-dc8d2563cdb5-kube-api-access-4dvx8\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6\" (UID: \"359a6ecd-500d-4916-a2bb-dc8d2563cdb5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"
Oct 30 18:52:58.496061 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.495946    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/359a6ecd-500d-4916-a2bb-dc8d2563cdb5-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6\" (UID: \"359a6ecd-500d-4916-a2bb-dc8d2563cdb5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"
Oct 30 18:52:58.496280 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.496263    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/359a6ecd-500d-4916-a2bb-dc8d2563cdb5-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6\" (UID: \"359a6ecd-500d-4916-a2bb-dc8d2563cdb5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"
Oct 30 18:52:58.555281 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.555250    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dvx8\" (UniqueName: \"kubernetes.io/projected/359a6ecd-500d-4916-a2bb-dc8d2563cdb5-kube-api-access-4dvx8\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6\" (UID: \"359a6ecd-500d-4916-a2bb-dc8d2563cdb5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"
Oct 30 18:52:58.653445 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:58.653412    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"
Oct 30 18:52:59.086722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:59.086688    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6"]
Oct 30 18:52:59.094096 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:52:59.094069    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod359a6ecd_500d_4916_a2bb_dc8d2563cdb5.slice/crio-9266a00a2e9ca55f494a2c797712ec6861be34e1a74a7e9713fdfbc058bfc4b5 WatchSource:0}: Error finding container 9266a00a2e9ca55f494a2c797712ec6861be34e1a74a7e9713fdfbc058bfc4b5: Status 404 returned error can't find the container with id 9266a00a2e9ca55f494a2c797712ec6861be34e1a74a7e9713fdfbc058bfc4b5
Oct 30 18:52:59.251373 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:52:59.251339    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6" event={"ID":"359a6ecd-500d-4916-a2bb-dc8d2563cdb5","Type":"ContainerStarted","Data":"9266a00a2e9ca55f494a2c797712ec6861be34e1a74a7e9713fdfbc058bfc4b5"}
Oct 30 18:53:00.257464 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:00.257370    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj" event={"ID":"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a","Type":"ContainerStarted","Data":"c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81"}
Oct 30 18:53:00.257464 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:00.257445    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:53:00.468753 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:00.468696    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj" podStartSLOduration=2.272813261 podStartE2EDuration="7.468681775s" totalImagesPullingTime="5.195868514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:00.46819026 +0000 UTC m=+373.052673254" watchObservedRunningTime="2026-10-30 18:53:00.468681775 +0000 UTC m=+373.053164777"
Oct 30 18:53:01.214644 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:01.214606    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-jd8zk"
Oct 30 18:53:02.268227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:02.268135    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6" event={"ID":"359a6ecd-500d-4916-a2bb-dc8d2563cdb5","Type":"ContainerStarted","Data":"949d66de4c65df232dd0ceea7eebb3c85d7fddb2ff9981cb70336a460fcf9ff5"}
Oct 30 18:53:02.483968 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:02.483901    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-gwgm6" podStartSLOduration=1.716244921 podStartE2EDuration="4.483887672s" totalImagesPullingTime="2.767642751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:02.436208029 +0000 UTC m=+375.020691028" watchObservedRunningTime="2026-10-30 18:53:02.483887672 +0000 UTC m=+375.068370668"
Oct 30 18:53:05.667301 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.667263    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"]
Oct 30 18:53:05.671515 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.671489    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.722288 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.722251    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 30 18:53:05.752168 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.752135    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.752312 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.752180    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npl59\" (UniqueName: \"kubernetes.io/projected/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-kube-api-access-npl59\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.752312 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.752272    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.770791 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.770765    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ptp75\""
Oct 30 18:53:05.770791 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.770767    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 30 18:53:05.772251 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.772234    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"]
Oct 30 18:53:05.853480 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.853451    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.853639 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.853503    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.853639 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.853538    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npl59\" (UniqueName: \"kubernetes.io/projected/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-kube-api-access-npl59\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.854161 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.854138    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.855948 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.855927    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.912112 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.912083    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npl59\" (UniqueName: \"kubernetes.io/projected/1fa31cdb-42cf-4f45-a06a-aab92e1f0558-kube-api-access-npl59\") pod \"kuadrant-console-plugin-66c4fb575d-2kfmd\" (UID: \"1fa31cdb-42cf-4f45-a06a-aab92e1f0558\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:05.982509 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:05.982429    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"
Oct 30 18:53:06.367948 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:06.367908    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd"]
Oct 30 18:53:06.374153 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:53:06.374125    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fa31cdb_42cf_4f45_a06a_aab92e1f0558.slice/crio-03076ea1f01c0d2cb16e3ce31f87a9cc07cf7ba5217ca42c611ef0ddc7e914fd WatchSource:0}: Error finding container 03076ea1f01c0d2cb16e3ce31f87a9cc07cf7ba5217ca42c611ef0ddc7e914fd: Status 404 returned error can't find the container with id 03076ea1f01c0d2cb16e3ce31f87a9cc07cf7ba5217ca42c611ef0ddc7e914fd
Oct 30 18:53:07.291482 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:07.291449    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd" event={"ID":"1fa31cdb-42cf-4f45-a06a-aab92e1f0558","Type":"ContainerStarted","Data":"03076ea1f01c0d2cb16e3ce31f87a9cc07cf7ba5217ca42c611ef0ddc7e914fd"}
Oct 30 18:53:11.265352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:11.265312    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:53:12.316903 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:12.316869    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd" event={"ID":"1fa31cdb-42cf-4f45-a06a-aab92e1f0558","Type":"ContainerStarted","Data":"65d68d4ecab8ba77056e24e13aa073aa28dd90d3b2d6c9387e73dba2c8f77c27"}
Oct 30 18:53:12.432532 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:12.432475    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2kfmd" podStartSLOduration=1.590798308 podStartE2EDuration="7.432462407s" totalImagesPullingTime="5.841664099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:12.431632791 +0000 UTC m=+385.016115790" watchObservedRunningTime="2026-10-30 18:53:12.432462407 +0000 UTC m=+385.016945404"
Oct 30 18:53:16.186522 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:53:16.186482    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:53:19.785559 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.785520    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-42zpw"]
Oct 30 18:53:19.792087 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.792060    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:19.842371 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.842339    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 18:53:19.848877 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.848850    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-42zpw"]
Oct 30 18:53:19.871269 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.871231    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7f85d81-ce68-4918-a00d-db59720b7651-config-file\") pod \"limitador-limitador-74dfcfb894-42zpw\" (UID: \"c7f85d81-ce68-4918-a00d-db59720b7651\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:19.871437 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.871273    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5kv2\" (UniqueName: \"kubernetes.io/projected/c7f85d81-ce68-4918-a00d-db59720b7651-kube-api-access-v5kv2\") pod \"limitador-limitador-74dfcfb894-42zpw\" (UID: \"c7f85d81-ce68-4918-a00d-db59720b7651\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:19.971736 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.971700    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7f85d81-ce68-4918-a00d-db59720b7651-config-file\") pod \"limitador-limitador-74dfcfb894-42zpw\" (UID: \"c7f85d81-ce68-4918-a00d-db59720b7651\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:19.971736 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.971744    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5kv2\" (UniqueName: \"kubernetes.io/projected/c7f85d81-ce68-4918-a00d-db59720b7651-kube-api-access-v5kv2\") pod \"limitador-limitador-74dfcfb894-42zpw\" (UID: \"c7f85d81-ce68-4918-a00d-db59720b7651\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:19.972625 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:19.972599    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7f85d81-ce68-4918-a00d-db59720b7651-config-file\") pod \"limitador-limitador-74dfcfb894-42zpw\" (UID: \"c7f85d81-ce68-4918-a00d-db59720b7651\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:20.010616 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.010582    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-42zpw"]
Oct 30 18:53:20.031388 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.031354    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5kv2\" (UniqueName: \"kubernetes.io/projected/c7f85d81-ce68-4918-a00d-db59720b7651-kube-api-access-v5kv2\") pod \"limitador-limitador-74dfcfb894-42zpw\" (UID: \"c7f85d81-ce68-4918-a00d-db59720b7651\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:20.104417 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.104368    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:20.479994 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.479912    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-42zpw"]
Oct 30 18:53:20.486724 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:53:20.486695    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f85d81_ce68_4918_a00d_db59720b7651.slice/crio-9fd0257c78b1e578b23f1685ed4a9f5d8a10bbfee9e3ef0d7148a90a880929ad WatchSource:0}: Error finding container 9fd0257c78b1e578b23f1685ed4a9f5d8a10bbfee9e3ef0d7148a90a880929ad: Status 404 returned error can't find the container with id 9fd0257c78b1e578b23f1685ed4a9f5d8a10bbfee9e3ef0d7148a90a880929ad
Oct 30 18:53:20.571181 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.571146    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"]
Oct 30 18:53:20.578202 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.578176    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.628925 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.628886    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4vwzh\""
Oct 30 18:53:20.637390 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.637356    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"]
Oct 30 18:53:20.681577 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681537    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.681763 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681582    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.681763 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681608    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.681888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681773    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b17c47fb-fc48-4456-a64f-06ec2d144842-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.681888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681846    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k974\" (UniqueName: \"kubernetes.io/projected/b17c47fb-fc48-4456-a64f-06ec2d144842-kube-api-access-8k974\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.681888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681873    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.681986 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681893    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.681986 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681912    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.682049 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.681997    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783224 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783131    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783224 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783173    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783224 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783207    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783508 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783254    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783508 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783283    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783508 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783305    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783508 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783357    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b17c47fb-fc48-4456-a64f-06ec2d144842-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783508 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783399    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k974\" (UniqueName: \"kubernetes.io/projected/b17c47fb-fc48-4456-a64f-06ec2d144842-kube-api-access-8k974\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783508 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783428    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783809 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783614    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.783943 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.783914    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.784064 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.784039    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.784147 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.784039    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.784288 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.784267    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b17c47fb-fc48-4456-a64f-06ec2d144842-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.785570 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.785545    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.785894 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.785660    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.842727 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.842694    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b17c47fb-fc48-4456-a64f-06ec2d144842-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.842964 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.842945    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k974\" (UniqueName: \"kubernetes.io/projected/b17c47fb-fc48-4456-a64f-06ec2d144842-kube-api-access-8k974\") pod \"data-science-gateway-data-science-gateway-class-67b5c995fc6lch7\" (UID: \"b17c47fb-fc48-4456-a64f-06ec2d144842\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:20.892012 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:20.891964    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:21.173326 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.173292    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-zzmfk"]
Oct 30 18:53:21.178506 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.178484    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-zzmfk"
Oct 30 18:53:21.228628 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.228592    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gcrvt\""
Oct 30 18:53:21.232983 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.232959    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-zzmfk"]
Oct 30 18:53:21.286303 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.286270    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s44g8\" (UniqueName: \"kubernetes.io/projected/be2dc26d-2ee6-40b6-9d40-792465445a70-kube-api-access-s44g8\") pod \"authorino-57f7f8659f-zzmfk\" (UID: \"be2dc26d-2ee6-40b6-9d40-792465445a70\") " pod="kuadrant-system/authorino-57f7f8659f-zzmfk"
Oct 30 18:53:21.317276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.317244    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"]
Oct 30 18:53:21.322537 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:53:21.322504    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb17c47fb_fc48_4456_a64f_06ec2d144842.slice/crio-005b1ca68a93b912cb640e7dc89aa196688faee465874e596518059baf5954bd WatchSource:0}: Error finding container 005b1ca68a93b912cb640e7dc89aa196688faee465874e596518059baf5954bd: Status 404 returned error can't find the container with id 005b1ca68a93b912cb640e7dc89aa196688faee465874e596518059baf5954bd
Oct 30 18:53:21.324718 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.324682    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:53:21.324794 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.324750    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:53:21.324794 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.324779    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:53:21.361226 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.361195    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw" event={"ID":"c7f85d81-ce68-4918-a00d-db59720b7651","Type":"ContainerStarted","Data":"9fd0257c78b1e578b23f1685ed4a9f5d8a10bbfee9e3ef0d7148a90a880929ad"}
Oct 30 18:53:21.362455 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.362418    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7" event={"ID":"b17c47fb-fc48-4456-a64f-06ec2d144842","Type":"ContainerStarted","Data":"005b1ca68a93b912cb640e7dc89aa196688faee465874e596518059baf5954bd"}
Oct 30 18:53:21.387058 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.387025    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s44g8\" (UniqueName: \"kubernetes.io/projected/be2dc26d-2ee6-40b6-9d40-792465445a70-kube-api-access-s44g8\") pod \"authorino-57f7f8659f-zzmfk\" (UID: \"be2dc26d-2ee6-40b6-9d40-792465445a70\") " pod="kuadrant-system/authorino-57f7f8659f-zzmfk"
Oct 30 18:53:21.446197 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.446122    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s44g8\" (UniqueName: \"kubernetes.io/projected/be2dc26d-2ee6-40b6-9d40-792465445a70-kube-api-access-s44g8\") pod \"authorino-57f7f8659f-zzmfk\" (UID: \"be2dc26d-2ee6-40b6-9d40-792465445a70\") " pod="kuadrant-system/authorino-57f7f8659f-zzmfk"
Oct 30 18:53:21.491053 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.490765    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-zzmfk"
Oct 30 18:53:21.865517 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:21.865482    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-zzmfk"]
Oct 30 18:53:21.870843 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:53:21.870792    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe2dc26d_2ee6_40b6_9d40_792465445a70.slice/crio-3c7c95c93e784e3e197f79a0f653a67ecc58a6d5cd7c2bf366464821d4ea8a81 WatchSource:0}: Error finding container 3c7c95c93e784e3e197f79a0f653a67ecc58a6d5cd7c2bf366464821d4ea8a81: Status 404 returned error can't find the container with id 3c7c95c93e784e3e197f79a0f653a67ecc58a6d5cd7c2bf366464821d4ea8a81
Oct 30 18:53:22.368263 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:22.368225    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-zzmfk" event={"ID":"be2dc26d-2ee6-40b6-9d40-792465445a70","Type":"ContainerStarted","Data":"3c7c95c93e784e3e197f79a0f653a67ecc58a6d5cd7c2bf366464821d4ea8a81"}
Oct 30 18:53:22.369881 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:22.369855    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7" event={"ID":"b17c47fb-fc48-4456-a64f-06ec2d144842","Type":"ContainerStarted","Data":"ad3dff560129d09bff5d0bf07d558e9ac97a21ec0b67415811d6fa56248995d2"}
Oct 30 18:53:22.488249 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:22.488192    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7" podStartSLOduration=2.488176198 podStartE2EDuration="2.488176198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:22.486673008 +0000 UTC m=+395.071156007" watchObservedRunningTime="2026-10-30 18:53:22.488176198 +0000 UTC m=+395.072659194"
Oct 30 18:53:22.892247 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:22.892208    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:22.897374 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:22.897345    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:23.377948 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.377911    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw" event={"ID":"c7f85d81-ce68-4918-a00d-db59720b7651","Type":"ContainerStarted","Data":"56d94f3bcbcb92403251c44789cc9a41aa5fc3c14927f9819a8f43feef40bd2f"}
Oct 30 18:53:23.378160 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.378004    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:23.378301 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.378283    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:23.379258 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.379240    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67b5c995fc6lch7"
Oct 30 18:53:23.490807 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.490749    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw" podStartSLOduration=2.221710512 podStartE2EDuration="4.490735999s" totalImagesPullingTime="2.269025487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:23.489646197 +0000 UTC m=+396.074129193" watchObservedRunningTime="2026-10-30 18:53:23.490735999 +0000 UTC m=+396.075218993"
Oct 30 18:53:23.829011 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.828929    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-mgd48"]
Oct 30 18:53:23.832547 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.832520    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48"
Oct 30 18:53:23.894057 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.894013    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-mgd48"]
Oct 30 18:53:23.908548 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:23.908512    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8qdt\" (UniqueName: \"kubernetes.io/projected/8b0cf4b1-708b-484d-8497-d00966e3e270-kube-api-access-b8qdt\") pod \"authorino-56c6c9ddd5-mgd48\" (UID: \"8b0cf4b1-708b-484d-8497-d00966e3e270\") " pod="kuadrant-system/authorino-56c6c9ddd5-mgd48"
Oct 30 18:53:24.009766 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:24.009731    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qdt\" (UniqueName: \"kubernetes.io/projected/8b0cf4b1-708b-484d-8497-d00966e3e270-kube-api-access-b8qdt\") pod \"authorino-56c6c9ddd5-mgd48\" (UID: \"8b0cf4b1-708b-484d-8497-d00966e3e270\") " pod="kuadrant-system/authorino-56c6c9ddd5-mgd48"
Oct 30 18:53:24.068633 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:24.068603    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8qdt\" (UniqueName: \"kubernetes.io/projected/8b0cf4b1-708b-484d-8497-d00966e3e270-kube-api-access-b8qdt\") pod \"authorino-56c6c9ddd5-mgd48\" (UID: \"8b0cf4b1-708b-484d-8497-d00966e3e270\") " pod="kuadrant-system/authorino-56c6c9ddd5-mgd48"
Oct 30 18:53:24.144911 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:24.144870    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48"
Oct 30 18:53:24.522551 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:24.522479    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-mgd48"]
Oct 30 18:53:24.528410 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:53:24.528378    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b0cf4b1_708b_484d_8497_d00966e3e270.slice/crio-15d518020f1e94997cf612f04c869f1507231954321d32cfc3c0a062385c12db WatchSource:0}: Error finding container 15d518020f1e94997cf612f04c869f1507231954321d32cfc3c0a062385c12db: Status 404 returned error can't find the container with id 15d518020f1e94997cf612f04c869f1507231954321d32cfc3c0a062385c12db
Oct 30 18:53:25.387479 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:25.387435    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48" event={"ID":"8b0cf4b1-708b-484d-8497-d00966e3e270","Type":"ContainerStarted","Data":"15d518020f1e94997cf612f04c869f1507231954321d32cfc3c0a062385c12db"}
Oct 30 18:53:26.395838 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:26.393779    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-zzmfk" event={"ID":"be2dc26d-2ee6-40b6-9d40-792465445a70","Type":"ContainerStarted","Data":"c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c"}
Oct 30 18:53:26.395838 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:26.395597    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48" event={"ID":"8b0cf4b1-708b-484d-8497-d00966e3e270","Type":"ContainerStarted","Data":"6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d"}
Oct 30 18:53:26.514139 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:26.514089    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-zzmfk" podStartSLOduration=1.949594207 podStartE2EDuration="5.514073595s" totalImagesPullingTime="3.564479388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:26.513294507 +0000 UTC m=+399.097777503" watchObservedRunningTime="2026-10-30 18:53:26.514073595 +0000 UTC m=+399.098556591"
Oct 30 18:53:26.633829 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:26.633755    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48" podStartSLOduration=2.226523132 podStartE2EDuration="3.633740602s" totalImagesPullingTime="1.40721747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:26.631190002 +0000 UTC m=+399.215672999" watchObservedRunningTime="2026-10-30 18:53:26.633740602 +0000 UTC m=+399.218223599"
Oct 30 18:53:26.666507 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:26.666430    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-zzmfk"]
Oct 30 18:53:28.405132 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:28.405068    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-zzmfk" podUID="be2dc26d-2ee6-40b6-9d40-792465445a70" containerName="authorino" containerID="cri-o://c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c" gracePeriod=30
Oct 30 18:53:28.643120 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:28.643096    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-zzmfk"
Oct 30 18:53:28.749670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:28.749587    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be2dc26d-2ee6-40b6-9d40-792465445a70-kube-api-access-s44g8\" (UniqueName: \"kubernetes.io/projected/be2dc26d-2ee6-40b6-9d40-792465445a70-kube-api-access-s44g8\") pod \"be2dc26d-2ee6-40b6-9d40-792465445a70\" (UID: \"be2dc26d-2ee6-40b6-9d40-792465445a70\") "
Oct 30 18:53:28.751793 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:28.751768    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2dc26d-2ee6-40b6-9d40-792465445a70-kube-api-access-s44g8" pod "be2dc26d-2ee6-40b6-9d40-792465445a70" (UID: "be2dc26d-2ee6-40b6-9d40-792465445a70"). InnerVolumeSpecName "kube-api-access-s44g8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:53:28.850975 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:28.850945    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s44g8\" (UniqueName: \"kubernetes.io/projected/be2dc26d-2ee6-40b6-9d40-792465445a70-kube-api-access-s44g8\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:53:28.883526 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:28.883490    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"]
Oct 30 18:53:28.883730 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:28.883694    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj" podUID="e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a" containerName="manager" containerID="cri-o://c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81" gracePeriod=10
Oct 30 18:53:29.411303 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.411268    5232 generic.go:356] "Generic (PLEG): container finished" podID="be2dc26d-2ee6-40b6-9d40-792465445a70" containerID="c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c" exitCode=0
Oct 30 18:53:29.411902 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.411348    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-zzmfk"
Oct 30 18:53:29.411902 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.411348    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-zzmfk" event={"ID":"be2dc26d-2ee6-40b6-9d40-792465445a70","Type":"ContainerDied","Data":"c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c"}
Oct 30 18:53:29.411902 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.411391    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-zzmfk" event={"ID":"be2dc26d-2ee6-40b6-9d40-792465445a70","Type":"ContainerDied","Data":"3c7c95c93e784e3e197f79a0f653a67ecc58a6d5cd7c2bf366464821d4ea8a81"}
Oct 30 18:53:29.411902 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.411408    5232 scope.go:122] "RemoveContainer" containerID="c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c"
Oct 30 18:53:29.419765 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.419744    5232 scope.go:122] "RemoveContainer" containerID="c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c"
Oct 30 18:53:29.420040 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:53:29.420021    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c\": container with ID starting with c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c not found: ID does not exist" containerID="c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c"
Oct 30 18:53:29.420120 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.420049    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c"} err="failed to get container status \"c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c\": rpc error: code = NotFound desc = could not find container \"c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c\": container with ID starting with c7208d90d2126be855e6b38ef6c7e2ed734ad9f440aac3d41001e839831eac5c not found: ID does not exist"
Oct 30 18:53:29.594928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.594892    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-zzmfk"]
Oct 30 18:53:29.599807 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:29.599782    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-zzmfk"]
Oct 30 18:53:30.023060 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.023029    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:53:30.164223 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.164139    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-extensions-socket-volume\") pod \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\" (UID: \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\") "
Oct 30 18:53:30.164223 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.164208    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-kube-api-access-xgwk4\" (UniqueName: \"kubernetes.io/projected/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-kube-api-access-xgwk4\") pod \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\" (UID: \"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a\") "
Oct 30 18:53:30.164718 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.164690    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-extensions-socket-volume" pod "e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a" (UID: "e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:53:30.166326 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.166296    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-kube-api-access-xgwk4" pod "e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a" (UID: "e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a"). InnerVolumeSpecName "kube-api-access-xgwk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:53:30.265337 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.265301    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgwk4\" (UniqueName: \"kubernetes.io/projected/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-kube-api-access-xgwk4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:53:30.265337 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.265331    5232 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a-extensions-socket-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:53:30.418301 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.418128    5232 generic.go:356] "Generic (PLEG): container finished" podID="e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a" containerID="c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81" exitCode=0
Oct 30 18:53:30.418301 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.418190    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"
Oct 30 18:53:30.418301 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.418192    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj" event={"ID":"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a","Type":"ContainerDied","Data":"c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81"}
Oct 30 18:53:30.418301 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.418289    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj" event={"ID":"e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a","Type":"ContainerDied","Data":"52b064c39a6b489b17dea41a9d40b784a1806585df91272fa7c302a16deb1a43"}
Oct 30 18:53:30.418929 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.418337    5232 scope.go:122] "RemoveContainer" containerID="c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81"
Oct 30 18:53:30.427193 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.427174    5232 scope.go:122] "RemoveContainer" containerID="c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81"
Oct 30 18:53:30.427462 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:53:30.427443    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81\": container with ID starting with c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81 not found: ID does not exist" containerID="c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81"
Oct 30 18:53:30.427503 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.427470    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81"} err="failed to get container status \"c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81\": rpc error: code = NotFound desc = could not find container \"c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81\": container with ID starting with c5c354d789d41189b780b893b5333a746bf5d8cd69ece1b5550c5c1b99692a81 not found: ID does not exist"
Oct 30 18:53:30.577089 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.577057    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"]
Oct 30 18:53:30.581502 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:30.581478    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-t7tkj"]
Oct 30 18:53:31.537738 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:31.537704    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be2dc26d-2ee6-40b6-9d40-792465445a70" path="/var/lib/kubelet/pods/be2dc26d-2ee6-40b6-9d40-792465445a70/volumes"
Oct 30 18:53:31.538159 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:31.538082    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a" path="/var/lib/kubelet/pods/e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a/volumes"
Oct 30 18:53:32.143293 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.143259    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"]
Oct 30 18:53:32.143577 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.143564    5232 cpu_manager.go:414] "containerMap: removing container" podUID="be2dc26d-2ee6-40b6-9d40-792465445a70" containerName="authorino"
Oct 30 18:53:32.143621 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.143578    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be2dc26d-2ee6-40b6-9d40-792465445a70" containerName="authorino"
Oct 30 18:53:32.143621 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.143594    5232 cpu_manager.go:414] "containerMap: removing container" podUID="e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a" containerName="manager"
Oct 30 18:53:32.143621 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.143599    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a" containerName="manager"
Oct 30 18:53:32.143728 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.143649    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5726fe0-afd3-4abf-a89e-9b9ea85bbe4a" containerName="manager"
Oct 30 18:53:32.143728 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.143662    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="be2dc26d-2ee6-40b6-9d40-792465445a70" containerName="authorino"
Oct 30 18:53:32.175828 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.175775    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.226254 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.226214    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-wgfr7\""
Oct 30 18:53:32.275197 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.275167    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"]
Oct 30 18:53:32.277009 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.276989    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 18:53:32.277136 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.277115    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 18:53:32.281630 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.281611    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3f10fb-bb7e-464a-be03-8553bf46f443-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.281681 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.281648    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qm7n\" (UniqueName: \"kubernetes.io/projected/0b3f10fb-bb7e-464a-be03-8553bf46f443-kube-api-access-5qm7n\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.281753 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.281736    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b3f10fb-bb7e-464a-be03-8553bf46f443-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.281836 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.281801    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b3f10fb-bb7e-464a-be03-8553bf46f443-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.372412 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.372374    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 18:53:32.382682 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.382647    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b3f10fb-bb7e-464a-be03-8553bf46f443-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.382682 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.382679    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b3f10fb-bb7e-464a-be03-8553bf46f443-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.382888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.382718    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3f10fb-bb7e-464a-be03-8553bf46f443-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.382888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.382741    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qm7n\" (UniqueName: \"kubernetes.io/projected/0b3f10fb-bb7e-464a-be03-8553bf46f443-kube-api-access-5qm7n\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.383145 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.383124    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3f10fb-bb7e-464a-be03-8553bf46f443-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.383180 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.383120    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b3f10fb-bb7e-464a-be03-8553bf46f443-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.383610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.383592    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b3f10fb-bb7e-464a-be03-8553bf46f443-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.441775 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.441693    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qm7n\" (UniqueName: \"kubernetes.io/projected/0b3f10fb-bb7e-464a-be03-8553bf46f443-kube-api-access-5qm7n\") pod \"kserve-module-controller-manager-7566cf5fc8-5t7c4\" (UID: \"0b3f10fb-bb7e-464a-be03-8553bf46f443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.511232 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.511198    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:32.945200 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:32.945165    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"]
Oct 30 18:53:32.951281 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:53:32.951256    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b3f10fb_bb7e_464a_be03_8553bf46f443.slice/crio-3a0bafae93a73f35e82240002349d7f102d802a443d2fbf41cfdccf45dabf533 WatchSource:0}: Error finding container 3a0bafae93a73f35e82240002349d7f102d802a443d2fbf41cfdccf45dabf533: Status 404 returned error can't find the container with id 3a0bafae93a73f35e82240002349d7f102d802a443d2fbf41cfdccf45dabf533
Oct 30 18:53:33.434027 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:33.433988    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4" event={"ID":"0b3f10fb-bb7e-464a-be03-8553bf46f443","Type":"ContainerStarted","Data":"3a0bafae93a73f35e82240002349d7f102d802a443d2fbf41cfdccf45dabf533"}
Oct 30 18:53:34.383412 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:34.383379    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:36.449680 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:36.449643    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4" event={"ID":"0b3f10fb-bb7e-464a-be03-8553bf46f443","Type":"ContainerStarted","Data":"57e0c8b4c4d463c5f491339113e671196eef5e1e85efc1cab2458244c5c7c0f0"}
Oct 30 18:53:36.450117 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:36.449705    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:36.612255 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:36.612193    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4" podStartSLOduration=1.71978623 podStartE2EDuration="4.612175175s" totalImagesPullingTime="2.892388945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:36.611736946 +0000 UTC m=+409.196219958" watchObservedRunningTime="2026-10-30 18:53:36.612175175 +0000 UTC m=+409.196658173"
Oct 30 18:53:39.798087 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:39.798050    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-42zpw"]
Oct 30 18:53:39.798482 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:39.798244    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw" podUID="c7f85d81-ce68-4918-a00d-db59720b7651" containerName="limitador" containerID="cri-o://56d94f3bcbcb92403251c44789cc9a41aa5fc3c14927f9819a8f43feef40bd2f" gracePeriod=30
Oct 30 18:53:40.469338 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.469253    5232 generic.go:356] "Generic (PLEG): container finished" podID="c7f85d81-ce68-4918-a00d-db59720b7651" containerID="56d94f3bcbcb92403251c44789cc9a41aa5fc3c14927f9819a8f43feef40bd2f" exitCode=0
Oct 30 18:53:40.469490 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.469333    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw" event={"ID":"c7f85d81-ce68-4918-a00d-db59720b7651","Type":"ContainerDied","Data":"56d94f3bcbcb92403251c44789cc9a41aa5fc3c14927f9819a8f43feef40bd2f"}
Oct 30 18:53:40.734399 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.734374    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:40.869492 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.869454    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7f85d81-ce68-4918-a00d-db59720b7651-config-file\" (UniqueName: \"kubernetes.io/configmap/c7f85d81-ce68-4918-a00d-db59720b7651-config-file\") pod \"c7f85d81-ce68-4918-a00d-db59720b7651\" (UID: \"c7f85d81-ce68-4918-a00d-db59720b7651\") "
Oct 30 18:53:40.870021 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.869567    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7f85d81-ce68-4918-a00d-db59720b7651-kube-api-access-v5kv2\" (UniqueName: \"kubernetes.io/projected/c7f85d81-ce68-4918-a00d-db59720b7651-kube-api-access-v5kv2\") pod \"c7f85d81-ce68-4918-a00d-db59720b7651\" (UID: \"c7f85d81-ce68-4918-a00d-db59720b7651\") "
Oct 30 18:53:40.870021 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.869865    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7f85d81-ce68-4918-a00d-db59720b7651-config-file" pod "c7f85d81-ce68-4918-a00d-db59720b7651" (UID: "c7f85d81-ce68-4918-a00d-db59720b7651"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:53:40.871625 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.871602    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f85d81-ce68-4918-a00d-db59720b7651-kube-api-access-v5kv2" pod "c7f85d81-ce68-4918-a00d-db59720b7651" (UID: "c7f85d81-ce68-4918-a00d-db59720b7651"). InnerVolumeSpecName "kube-api-access-v5kv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:53:40.971200 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.971119    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5kv2\" (UniqueName: \"kubernetes.io/projected/c7f85d81-ce68-4918-a00d-db59720b7651-kube-api-access-v5kv2\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:53:40.971200 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:40.971150    5232 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7f85d81-ce68-4918-a00d-db59720b7651-config-file\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:53:41.474887 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:41.474859    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw"
Oct 30 18:53:41.475060 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:41.474852    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-42zpw" event={"ID":"c7f85d81-ce68-4918-a00d-db59720b7651","Type":"ContainerDied","Data":"9fd0257c78b1e578b23f1685ed4a9f5d8a10bbfee9e3ef0d7148a90a880929ad"}
Oct 30 18:53:41.475060 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:41.474982    5232 scope.go:122] "RemoveContainer" containerID="56d94f3bcbcb92403251c44789cc9a41aa5fc3c14927f9819a8f43feef40bd2f"
Oct 30 18:53:41.632521 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:41.632486    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-42zpw"]
Oct 30 18:53:41.680078 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:41.680046    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-42zpw"]
Oct 30 18:53:43.537285 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:43.537250    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7f85d81-ce68-4918-a00d-db59720b7651" path="/var/lib/kubelet/pods/c7f85d81-ce68-4918-a00d-db59720b7651/volumes"
Oct 30 18:53:46.186657 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:53:46.186617    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:53:50.818711 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.818671    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"]
Oct 30 18:53:50.819165 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.818996    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c7f85d81-ce68-4918-a00d-db59720b7651" containerName="limitador"
Oct 30 18:53:50.819165 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.819008    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7f85d81-ce68-4918-a00d-db59720b7651" containerName="limitador"
Oct 30 18:53:50.819165 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.819063    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7f85d81-ce68-4918-a00d-db59720b7651" containerName="limitador"
Oct 30 18:53:50.840341 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.840307    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.852414 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.852380    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64df7501-3a39-4c5c-a413-f0aadb87cb34-cert\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.852549 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.852484    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfthk\" (UniqueName: \"kubernetes.io/projected/64df7501-3a39-4c5c-a413-f0aadb87cb34-kube-api-access-zfthk\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.852598 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.852551    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64df7501-3a39-4c5c-a413-f0aadb87cb34-metrics-certs\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.852640 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.852620    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/64df7501-3a39-4c5c-a413-f0aadb87cb34-manager-config\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.885399 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.885358    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"]
Oct 30 18:53:50.919953 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.919916    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 30 18:53:50.931421 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.931400    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 30 18:53:50.938329 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.938302    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-52rs7\""
Oct 30 18:53:50.938429 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.938412    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 30 18:53:50.953884 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.953861    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64df7501-3a39-4c5c-a413-f0aadb87cb34-metrics-certs\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.953953 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.953892    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/64df7501-3a39-4c5c-a413-f0aadb87cb34-manager-config\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.953953 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.953919    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64df7501-3a39-4c5c-a413-f0aadb87cb34-cert\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.954060 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.953973    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfthk\" (UniqueName: \"kubernetes.io/projected/64df7501-3a39-4c5c-a413-f0aadb87cb34-kube-api-access-zfthk\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.954574 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.954554    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/64df7501-3a39-4c5c-a413-f0aadb87cb34-manager-config\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.956547 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.956524    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64df7501-3a39-4c5c-a413-f0aadb87cb34-metrics-certs\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:50.956547 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:50.956536    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64df7501-3a39-4c5c-a413-f0aadb87cb34-cert\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:51.013050 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:51.013020    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfthk\" (UniqueName: \"kubernetes.io/projected/64df7501-3a39-4c5c-a413-f0aadb87cb34-kube-api-access-zfthk\") pod \"jobset-controller-manager-dd6c4cbcb-v8g2g\" (UID: \"64df7501-3a39-4c5c-a413-f0aadb87cb34\") " pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:51.152378 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:51.152342    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:51.520252 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:51.520172    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"]
Oct 30 18:53:51.527030 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:53:51.527002    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64df7501_3a39_4c5c_a413_f0aadb87cb34.slice/crio-9446c73501c4bf03e0192fb7d8e7f56b1556c167a7ef54a18eaa194662d9654f WatchSource:0}: Error finding container 9446c73501c4bf03e0192fb7d8e7f56b1556c167a7ef54a18eaa194662d9654f: Status 404 returned error can't find the container with id 9446c73501c4bf03e0192fb7d8e7f56b1556c167a7ef54a18eaa194662d9654f
Oct 30 18:53:52.533590 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:52.533555    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g" event={"ID":"64df7501-3a39-4c5c-a413-f0aadb87cb34","Type":"ContainerStarted","Data":"9446c73501c4bf03e0192fb7d8e7f56b1556c167a7ef54a18eaa194662d9654f"}
Oct 30 18:53:56.423417 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.423382    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"]
Oct 30 18:53:56.430776 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.430750    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:53:56.485885 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.485834    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 18:53:56.485885 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.485862    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fkrtz\""
Oct 30 18:53:56.485885 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.485862    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 18:53:56.486175 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.485907    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 18:53:56.487735 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.487715    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"]
Oct 30 18:53:56.498640 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.498606    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5a4394b5-1f92-42de-85c6-b237231ba477-data\") pod \"postgres-7cdbf6dc44-wbfgw\" (UID: \"5a4394b5-1f92-42de-85c6-b237231ba477\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:53:56.498802 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.498780    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfjwf\" (UniqueName: \"kubernetes.io/projected/5a4394b5-1f92-42de-85c6-b237231ba477-kube-api-access-bfjwf\") pod \"postgres-7cdbf6dc44-wbfgw\" (UID: \"5a4394b5-1f92-42de-85c6-b237231ba477\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:53:56.554786 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.554749    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g" event={"ID":"64df7501-3a39-4c5c-a413-f0aadb87cb34","Type":"ContainerStarted","Data":"d9b0faed752cc4e0db25cb1b0b7e86511883722fb9d30adb0e7cb2737e971123"}
Oct 30 18:53:56.554979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.554865    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:53:56.599405 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.599372    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfjwf\" (UniqueName: \"kubernetes.io/projected/5a4394b5-1f92-42de-85c6-b237231ba477-kube-api-access-bfjwf\") pod \"postgres-7cdbf6dc44-wbfgw\" (UID: \"5a4394b5-1f92-42de-85c6-b237231ba477\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:53:56.599596 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.599444    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5a4394b5-1f92-42de-85c6-b237231ba477-data\") pod \"postgres-7cdbf6dc44-wbfgw\" (UID: \"5a4394b5-1f92-42de-85c6-b237231ba477\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:53:56.599879 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.599858    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5a4394b5-1f92-42de-85c6-b237231ba477-data\") pod \"postgres-7cdbf6dc44-wbfgw\" (UID: \"5a4394b5-1f92-42de-85c6-b237231ba477\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:53:56.659062 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.659027    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfjwf\" (UniqueName: \"kubernetes.io/projected/5a4394b5-1f92-42de-85c6-b237231ba477-kube-api-access-bfjwf\") pod \"postgres-7cdbf6dc44-wbfgw\" (UID: \"5a4394b5-1f92-42de-85c6-b237231ba477\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:53:56.679639 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.679550    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g" podStartSLOduration=2.5450037500000002 podStartE2EDuration="6.679534663s" totalImagesPullingTime="4.134530913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:56.678569444 +0000 UTC m=+429.263052457" watchObservedRunningTime="2026-10-30 18:53:56.679534663 +0000 UTC m=+429.264017660"
Oct 30 18:53:56.744992 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:56.744960    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:53:57.118010 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:57.117975    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"]
Oct 30 18:53:57.123957 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:53:57.123926    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a4394b5_1f92_42de_85c6_b237231ba477.slice/crio-cdb8ad238e9a20653c8c5ee50ff4f21a335b5226f96b8463fd49c6f6d154df52 WatchSource:0}: Error finding container cdb8ad238e9a20653c8c5ee50ff4f21a335b5226f96b8463fd49c6f6d154df52: Status 404 returned error can't find the container with id cdb8ad238e9a20653c8c5ee50ff4f21a335b5226f96b8463fd49c6f6d154df52
Oct 30 18:53:57.455862 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:57.455759    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-5t7c4"
Oct 30 18:53:57.560346 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:53:57.560310    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw" event={"ID":"5a4394b5-1f92-42de-85c6-b237231ba477","Type":"ContainerStarted","Data":"cdb8ad238e9a20653c8c5ee50ff4f21a335b5226f96b8463fd49c6f6d154df52"}
Oct 30 18:54:04.597181 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:04.597145    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw" event={"ID":"5a4394b5-1f92-42de-85c6-b237231ba477","Type":"ContainerStarted","Data":"15904894fd8a81a240dda6908551155f3138a8f5abc21ba324b7e559acbee0c6"}
Oct 30 18:54:04.597626 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:04.597282    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:54:04.709931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:04.709876    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw" podStartSLOduration=2.187788197 podStartE2EDuration="8.709861966s" totalImagesPullingTime="6.522073769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:04.708385574 +0000 UTC m=+437.292868572" watchObservedRunningTime="2026-10-30 18:54:04.709861966 +0000 UTC m=+437.294344962"
Oct 30 18:54:07.567142 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:07.567103    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-dd6c4cbcb-v8g2g"
Oct 30 18:54:10.630505 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:10.630476    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wbfgw"
Oct 30 18:54:16.186664 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:54:16.186622    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:54:17.734592 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.734558    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c44f9d774-xm88r"]
Oct 30 18:54:17.743194 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.743167    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:17.793676 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.793641    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 18:54:17.841279 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.841235    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 18:54:17.841279 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.841236    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t5tnj\""
Oct 30 18:54:17.841536 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.841488    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 18:54:17.843906 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.843877    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c44f9d774-xm88r"]
Oct 30 18:54:17.893667 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.893634    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-metrics-certs\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:17.893872 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.893676    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmmtl\" (UniqueName: \"kubernetes.io/projected/6da89f0b-1427-4dc2-a681-5e7d141b0cda-kube-api-access-hmmtl\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:17.893927 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.893870    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-webhook-cert\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:17.994938 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.994848    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-metrics-certs\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:17.994938 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.994892    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmmtl\" (UniqueName: \"kubernetes.io/projected/6da89f0b-1427-4dc2-a681-5e7d141b0cda-kube-api-access-hmmtl\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:17.995129 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.994948    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-webhook-cert\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:17.997397 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.997369    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-metrics-certs\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:17.997503 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:17.997415    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-webhook-cert\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:18.054340 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.054304    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmmtl\" (UniqueName: \"kubernetes.io/projected/6da89f0b-1427-4dc2-a681-5e7d141b0cda-kube-api-access-hmmtl\") pod \"maas-controller-c44f9d774-xm88r\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") " pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:18.058971 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.058947    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:18.310419 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.310327    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7447664ff8-lfrzx"]
Oct 30 18:54:18.315769 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.315737    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.373902 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.373865    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7447664ff8-lfrzx"]
Oct 30 18:54:18.398143 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.398114    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r52s4\" (UniqueName: \"kubernetes.io/projected/9070de12-c761-46e4-a362-2365219f92eb-kube-api-access-r52s4\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.398308 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.398162    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9070de12-c761-46e4-a362-2365219f92eb-webhook-cert\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.398308 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.398182    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9070de12-c761-46e4-a362-2365219f92eb-metrics-certs\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.432328 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.432294    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c44f9d774-xm88r"]
Oct 30 18:54:18.473567 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:54:18.473535    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da89f0b_1427_4dc2_a681_5e7d141b0cda.slice/crio-95c8bbc916fd3f0ee8d00a2809e86ae563cb665741bbf0b86251f24893d0821d WatchSource:0}: Error finding container 95c8bbc916fd3f0ee8d00a2809e86ae563cb665741bbf0b86251f24893d0821d: Status 404 returned error can't find the container with id 95c8bbc916fd3f0ee8d00a2809e86ae563cb665741bbf0b86251f24893d0821d
Oct 30 18:54:18.499608 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.499579    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r52s4\" (UniqueName: \"kubernetes.io/projected/9070de12-c761-46e4-a362-2365219f92eb-kube-api-access-r52s4\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.499767 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.499633    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9070de12-c761-46e4-a362-2365219f92eb-webhook-cert\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.499767 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.499662    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9070de12-c761-46e4-a362-2365219f92eb-metrics-certs\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.502074 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.502050    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9070de12-c761-46e4-a362-2365219f92eb-metrics-certs\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.502156 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.502097    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9070de12-c761-46e4-a362-2365219f92eb-webhook-cert\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.559142 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.559105    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r52s4\" (UniqueName: \"kubernetes.io/projected/9070de12-c761-46e4-a362-2365219f92eb-kube-api-access-r52s4\") pod \"maas-controller-7447664ff8-lfrzx\" (UID: \"9070de12-c761-46e4-a362-2365219f92eb\") " pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.629072 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.629040    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:18.665149 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:18.665105    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c44f9d774-xm88r" event={"ID":"6da89f0b-1427-4dc2-a681-5e7d141b0cda","Type":"ContainerStarted","Data":"95c8bbc916fd3f0ee8d00a2809e86ae563cb665741bbf0b86251f24893d0821d"}
Oct 30 18:54:19.008681 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:19.007654    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7447664ff8-lfrzx"]
Oct 30 18:54:19.013104 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:54:19.013072    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9070de12_c761_46e4_a362_2365219f92eb.slice/crio-ba574911edfeee0a25612a644ef66009920d34751e4485d7874e2e1189011e70 WatchSource:0}: Error finding container ba574911edfeee0a25612a644ef66009920d34751e4485d7874e2e1189011e70: Status 404 returned error can't find the container with id ba574911edfeee0a25612a644ef66009920d34751e4485d7874e2e1189011e70
Oct 30 18:54:19.672424 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:19.672347    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7447664ff8-lfrzx" event={"ID":"9070de12-c761-46e4-a362-2365219f92eb","Type":"ContainerStarted","Data":"ba574911edfeee0a25612a644ef66009920d34751e4485d7874e2e1189011e70"}
Oct 30 18:54:22.690995 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:22.690962    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7447664ff8-lfrzx" event={"ID":"9070de12-c761-46e4-a362-2365219f92eb","Type":"ContainerStarted","Data":"2f0bff07b5d09ef6e560949a4c5cd69fa8f20f0ffb290d667b85bb87a614973a"}
Oct 30 18:54:22.691474 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:22.691124    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:22.692534 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:22.692512    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c44f9d774-xm88r" event={"ID":"6da89f0b-1427-4dc2-a681-5e7d141b0cda","Type":"ContainerStarted","Data":"d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18"}
Oct 30 18:54:22.692625 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:22.692566    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:22.808893 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:22.808838    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7447664ff8-lfrzx" podStartSLOduration=1.918781941 podStartE2EDuration="4.808824679s" totalImagesPullingTime="2.890042738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:22.807278097 +0000 UTC m=+455.391761094" watchObservedRunningTime="2026-10-30 18:54:22.808824679 +0000 UTC m=+455.393307666"
Oct 30 18:54:22.922451 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:22.922390    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-c44f9d774-xm88r" podStartSLOduration=2.5008217090000002 podStartE2EDuration="5.92237503s" totalImagesPullingTime="3.421553321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:22.921237106 +0000 UTC m=+455.505720103" watchObservedRunningTime="2026-10-30 18:54:22.92237503 +0000 UTC m=+455.506858026"
Oct 30 18:54:33.700482 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:33.700413    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:54:33.700869 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:33.700580    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:33.865804 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:33.865769    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c44f9d774-xm88r"]
Oct 30 18:54:33.866020 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:33.865973    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-c44f9d774-xm88r" podUID="6da89f0b-1427-4dc2-a681-5e7d141b0cda" containerName="manager" containerID="cri-o://d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18" gracePeriod=30
Oct 30 18:54:34.643923 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.643888    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:34.749598 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.749566    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-webhook-cert\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-webhook-cert\") pod \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") "
Oct 30 18:54:34.750167 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.749797    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-metrics-certs\") pod \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") "
Oct 30 18:54:34.750167 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.749872    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6da89f0b-1427-4dc2-a681-5e7d141b0cda-kube-api-access-hmmtl\" (UniqueName: \"kubernetes.io/projected/6da89f0b-1427-4dc2-a681-5e7d141b0cda-kube-api-access-hmmtl\") pod \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\" (UID: \"6da89f0b-1427-4dc2-a681-5e7d141b0cda\") "
Oct 30 18:54:34.750856 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.750794    5232 generic.go:356] "Generic (PLEG): container finished" podID="6da89f0b-1427-4dc2-a681-5e7d141b0cda" containerID="d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18" exitCode=0
Oct 30 18:54:34.750987 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.750865    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c44f9d774-xm88r"
Oct 30 18:54:34.750987 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.750869    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c44f9d774-xm88r" event={"ID":"6da89f0b-1427-4dc2-a681-5e7d141b0cda","Type":"ContainerDied","Data":"d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18"}
Oct 30 18:54:34.751104 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.750995    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c44f9d774-xm88r" event={"ID":"6da89f0b-1427-4dc2-a681-5e7d141b0cda","Type":"ContainerDied","Data":"95c8bbc916fd3f0ee8d00a2809e86ae563cb665741bbf0b86251f24893d0821d"}
Oct 30 18:54:34.751104 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.751010    5232 scope.go:122] "RemoveContainer" containerID="d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18"
Oct 30 18:54:34.752220 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.752162    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-metrics-certs" pod "6da89f0b-1427-4dc2-a681-5e7d141b0cda" (UID: "6da89f0b-1427-4dc2-a681-5e7d141b0cda"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:34.752276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.752211    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-webhook-cert" pod "6da89f0b-1427-4dc2-a681-5e7d141b0cda" (UID: "6da89f0b-1427-4dc2-a681-5e7d141b0cda"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:34.752276 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.752242    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da89f0b-1427-4dc2-a681-5e7d141b0cda-kube-api-access-hmmtl" pod "6da89f0b-1427-4dc2-a681-5e7d141b0cda" (UID: "6da89f0b-1427-4dc2-a681-5e7d141b0cda"). InnerVolumeSpecName "kube-api-access-hmmtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:34.768672 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.768645    5232 scope.go:122] "RemoveContainer" containerID="d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18"
Oct 30 18:54:34.769056 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:54:34.769030    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18\": container with ID starting with d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18 not found: ID does not exist" containerID="d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18"
Oct 30 18:54:34.769124 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.769068    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18"} err="failed to get container status \"d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18\": rpc error: code = NotFound desc = could not find container \"d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18\": container with ID starting with d12ef2bc8af97a433f4ac491bfe4515da18c4f357938125ba5911406f3248f18 not found: ID does not exist"
Oct 30 18:54:34.851124 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.851090    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:34.851124 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.851117    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmmtl\" (UniqueName: \"kubernetes.io/projected/6da89f0b-1427-4dc2-a681-5e7d141b0cda-kube-api-access-hmmtl\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:34.851124 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:34.851131    5232 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6da89f0b-1427-4dc2-a681-5e7d141b0cda-webhook-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:35.175234 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:35.174656    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c44f9d774-xm88r"]
Oct 30 18:54:35.218803 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:35.218771    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c44f9d774-xm88r"]
Oct 30 18:54:35.537803 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:35.537768    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6da89f0b-1427-4dc2-a681-5e7d141b0cda" path="/var/lib/kubelet/pods/6da89f0b-1427-4dc2-a681-5e7d141b0cda/volumes"
Oct 30 18:54:40.526960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.526927    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-6fh88"]
Oct 30 18:54:40.527364 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.527276    5232 cpu_manager.go:414] "containerMap: removing container" podUID="6da89f0b-1427-4dc2-a681-5e7d141b0cda" containerName="manager"
Oct 30 18:54:40.527364 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.527287    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6da89f0b-1427-4dc2-a681-5e7d141b0cda" containerName="manager"
Oct 30 18:54:40.527364 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.527350    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="6da89f0b-1427-4dc2-a681-5e7d141b0cda" containerName="manager"
Oct 30 18:54:40.533289 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.533263    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:40.593086 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.593044    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d459d5ca-7637-41a0-845a-311991240ae9-cert\") pod \"kserve-controller-manager-776cc66d47-6fh88\" (UID: \"d459d5ca-7637-41a0-845a-311991240ae9\") " pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:40.593264 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.593105    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdrv8\" (UniqueName: \"kubernetes.io/projected/d459d5ca-7637-41a0-845a-311991240ae9-kube-api-access-pdrv8\") pod \"kserve-controller-manager-776cc66d47-6fh88\" (UID: \"d459d5ca-7637-41a0-845a-311991240ae9\") " pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:40.631646 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.631619    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bml6b\""
Oct 30 18:54:40.631646 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.631628    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 18:54:40.633901 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.633879    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-6fh88"]
Oct 30 18:54:40.693739 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.693698    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d459d5ca-7637-41a0-845a-311991240ae9-cert\") pod \"kserve-controller-manager-776cc66d47-6fh88\" (UID: \"d459d5ca-7637-41a0-845a-311991240ae9\") " pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:40.693739 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.693746    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdrv8\" (UniqueName: \"kubernetes.io/projected/d459d5ca-7637-41a0-845a-311991240ae9-kube-api-access-pdrv8\") pod \"kserve-controller-manager-776cc66d47-6fh88\" (UID: \"d459d5ca-7637-41a0-845a-311991240ae9\") " pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:40.696531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.696503    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d459d5ca-7637-41a0-845a-311991240ae9-cert\") pod \"kserve-controller-manager-776cc66d47-6fh88\" (UID: \"d459d5ca-7637-41a0-845a-311991240ae9\") " pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:40.754449 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.754415    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdrv8\" (UniqueName: \"kubernetes.io/projected/d459d5ca-7637-41a0-845a-311991240ae9-kube-api-access-pdrv8\") pod \"kserve-controller-manager-776cc66d47-6fh88\" (UID: \"d459d5ca-7637-41a0-845a-311991240ae9\") " pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:40.844952 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:40.844913    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:41.215040 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:41.214962    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-6fh88"]
Oct 30 18:54:41.220941 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:54:41.220912    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd459d5ca_7637_41a0_845a_311991240ae9.slice/crio-507ea17fa67b52557beb235b80beb9c432e211fe3a27b4600414b5684c7ce44c WatchSource:0}: Error finding container 507ea17fa67b52557beb235b80beb9c432e211fe3a27b4600414b5684c7ce44c: Status 404 returned error can't find the container with id 507ea17fa67b52557beb235b80beb9c432e211fe3a27b4600414b5684c7ce44c
Oct 30 18:54:41.785345 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:41.785310    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88" event={"ID":"d459d5ca-7637-41a0-845a-311991240ae9","Type":"ContainerStarted","Data":"507ea17fa67b52557beb235b80beb9c432e211fe3a27b4600414b5684c7ce44c"}
Oct 30 18:54:44.800911 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:44.800873    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88" event={"ID":"d459d5ca-7637-41a0-845a-311991240ae9","Type":"ContainerStarted","Data":"f1e8b79bd1500a5c8b95ad262b5e2cae892c3a7465313b1384fbeeced95ce4c0"}
Oct 30 18:54:44.801402 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:44.800997    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:54:44.914082 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:44.914029    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88" podStartSLOduration=1.8105505050000001 podStartE2EDuration="4.91401628s" totalImagesPullingTime="3.103465775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:44.912591593 +0000 UTC m=+477.497074593" watchObservedRunningTime="2026-10-30 18:54:44.91401628 +0000 UTC m=+477.498499276"
Oct 30 18:54:46.186884 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:54:46.186851    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:54:57.309430 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.309394    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"]
Oct 30 18:54:57.317531 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.317502    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.373567 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.373531    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"]
Oct 30 18:54:57.430891 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.430858    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7w45v\""
Oct 30 18:54:57.430891 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.430858    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 18:54:57.431094 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.430908    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 18:54:57.432261 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.432239    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26hqk\" (UniqueName: \"kubernetes.io/projected/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-kube-api-access-26hqk\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.432351 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.432279    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-openshift-service-ca\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.432351 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.432306    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-metrics-certs\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.432423 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.432404    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-maas-api-tls\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.533414 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.533378    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26hqk\" (UniqueName: \"kubernetes.io/projected/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-kube-api-access-26hqk\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.533414 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.533424    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-openshift-service-ca\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.533671 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.533449    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-metrics-certs\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.533671 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.533479    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-maas-api-tls\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.534117 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.534084    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-openshift-service-ca\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.535974 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.535948    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-maas-api-tls\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.536328 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.536310    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-metrics-certs\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.593664 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.593636    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26hqk\" (UniqueName: \"kubernetes.io/projected/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-kube-api-access-26hqk\") pod \"maas-api-764f864bb9-rwxr5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") " pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:57.630799 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:57.630770    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:54:58.005092 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:58.004997    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"]
Oct 30 18:54:58.010448 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:54:58.010410    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72dc80b9_c8d3_4cbf_8e98_a17fc64097c5.slice/crio-67b017e5cf8cf02d636f708af739ec4cd1f130b1665f66e0614a18fc2de44e8b WatchSource:0}: Error finding container 67b017e5cf8cf02d636f708af739ec4cd1f130b1665f66e0614a18fc2de44e8b: Status 404 returned error can't find the container with id 67b017e5cf8cf02d636f708af739ec4cd1f130b1665f66e0614a18fc2de44e8b
Oct 30 18:54:58.870647 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:54:58.870609    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5" event={"ID":"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5","Type":"ContainerStarted","Data":"67b017e5cf8cf02d636f708af739ec4cd1f130b1665f66e0614a18fc2de44e8b"}
Oct 30 18:55:00.881731 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:00.881698    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5" event={"ID":"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5","Type":"ContainerStarted","Data":"ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0"}
Oct 30 18:55:00.882176 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:00.881769    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:55:00.996947 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:00.996893    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5" podStartSLOduration=2.093016436 podStartE2EDuration="3.996880412s" totalImagesPullingTime="1.903863976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:00.995441631 +0000 UTC m=+493.579924627" watchObservedRunningTime="2026-10-30 18:55:00.996880412 +0000 UTC m=+493.581363408"
Oct 30 18:55:06.893116 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:06.893079    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:55:07.751839 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:07.740731    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-qcfqt"]
Oct 30 18:55:07.757835 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:07.757595    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt"
Oct 30 18:55:07.800624 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:07.800585    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-qcfqt"]
Oct 30 18:55:07.828402 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:07.828367    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmcn9\" (UniqueName: \"kubernetes.io/projected/ad673c6d-2077-4cdc-8628-4fda925c1517-kube-api-access-lmcn9\") pod \"authorino-6f6944bcdd-qcfqt\" (UID: \"ad673c6d-2077-4cdc-8628-4fda925c1517\") " pod="kuadrant-system/authorino-6f6944bcdd-qcfqt"
Oct 30 18:55:07.929502 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:07.929468    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmcn9\" (UniqueName: \"kubernetes.io/projected/ad673c6d-2077-4cdc-8628-4fda925c1517-kube-api-access-lmcn9\") pod \"authorino-6f6944bcdd-qcfqt\" (UID: \"ad673c6d-2077-4cdc-8628-4fda925c1517\") " pod="kuadrant-system/authorino-6f6944bcdd-qcfqt"
Oct 30 18:55:07.989179 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:07.989143    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmcn9\" (UniqueName: \"kubernetes.io/projected/ad673c6d-2077-4cdc-8628-4fda925c1517-kube-api-access-lmcn9\") pod \"authorino-6f6944bcdd-qcfqt\" (UID: \"ad673c6d-2077-4cdc-8628-4fda925c1517\") " pod="kuadrant-system/authorino-6f6944bcdd-qcfqt"
Oct 30 18:55:08.076687 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.076601    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt"
Oct 30 18:55:08.117845 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.117716    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-qcfqt"]
Oct 30 18:55:08.173764 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.173728    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6479bbd56d-kgdjl"]
Oct 30 18:55:08.180191 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.179266    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6479bbd56d-kgdjl"
Oct 30 18:55:08.212397 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.212367    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6479bbd56d-kgdjl"]
Oct 30 18:55:08.232756 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.232721    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59wj9\" (UniqueName: \"kubernetes.io/projected/01509dac-efd0-49cc-b923-a5c1835623e2-kube-api-access-59wj9\") pod \"authorino-6479bbd56d-kgdjl\" (UID: \"01509dac-efd0-49cc-b923-a5c1835623e2\") " pod="kuadrant-system/authorino-6479bbd56d-kgdjl"
Oct 30 18:55:08.334211 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.334127    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59wj9\" (UniqueName: \"kubernetes.io/projected/01509dac-efd0-49cc-b923-a5c1835623e2-kube-api-access-59wj9\") pod \"authorino-6479bbd56d-kgdjl\" (UID: \"01509dac-efd0-49cc-b923-a5c1835623e2\") " pod="kuadrant-system/authorino-6479bbd56d-kgdjl"
Oct 30 18:55:08.393152 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.393115    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59wj9\" (UniqueName: \"kubernetes.io/projected/01509dac-efd0-49cc-b923-a5c1835623e2-kube-api-access-59wj9\") pod \"authorino-6479bbd56d-kgdjl\" (UID: \"01509dac-efd0-49cc-b923-a5c1835623e2\") " pod="kuadrant-system/authorino-6479bbd56d-kgdjl"
Oct 30 18:55:08.462889 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.459932    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-qcfqt"]
Oct 30 18:55:08.466157 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.466131    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6479bbd56d-kgdjl"]
Oct 30 18:55:08.466392 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.466377    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6479bbd56d-kgdjl"
Oct 30 18:55:08.467026 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:55:08.467001    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad673c6d_2077_4cdc_8628_4fda925c1517.slice/crio-fee3693c453e256ee3aee4b5eef567eba821437f1d0ff79e8837feaac7f45b47 WatchSource:0}: Error finding container fee3693c453e256ee3aee4b5eef567eba821437f1d0ff79e8837feaac7f45b47: Status 404 returned error can't find the container with id fee3693c453e256ee3aee4b5eef567eba821437f1d0ff79e8837feaac7f45b47
Oct 30 18:55:08.495282 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.495249    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8558845494-2hrn5"]
Oct 30 18:55:08.499918 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.499893    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:55:08.565617 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.562658    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 18:55:08.565838 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.565727    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8558845494-2hrn5"]
Oct 30 18:55:08.636670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.636631    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qx6c\" (UniqueName: \"kubernetes.io/projected/04be873e-1f09-4680-be2a-5d7f3e7d5235-kube-api-access-6qx6c\") pod \"authorino-8558845494-2hrn5\" (UID: \"04be873e-1f09-4680-be2a-5d7f3e7d5235\") " pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:55:08.636898 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.636707    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/04be873e-1f09-4680-be2a-5d7f3e7d5235-tls-cert\") pod \"authorino-8558845494-2hrn5\" (UID: \"04be873e-1f09-4680-be2a-5d7f3e7d5235\") " pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:55:08.746584 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.746532    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/04be873e-1f09-4680-be2a-5d7f3e7d5235-tls-cert\") pod \"authorino-8558845494-2hrn5\" (UID: \"04be873e-1f09-4680-be2a-5d7f3e7d5235\") " pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:55:08.746778 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.746642    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qx6c\" (UniqueName: \"kubernetes.io/projected/04be873e-1f09-4680-be2a-5d7f3e7d5235-kube-api-access-6qx6c\") pod \"authorino-8558845494-2hrn5\" (UID: \"04be873e-1f09-4680-be2a-5d7f3e7d5235\") " pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:55:08.749470 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.749441    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/04be873e-1f09-4680-be2a-5d7f3e7d5235-tls-cert\") pod \"authorino-8558845494-2hrn5\" (UID: \"04be873e-1f09-4680-be2a-5d7f3e7d5235\") " pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:55:08.805791 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.805747    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qx6c\" (UniqueName: \"kubernetes.io/projected/04be873e-1f09-4680-be2a-5d7f3e7d5235-kube-api-access-6qx6c\") pod \"authorino-8558845494-2hrn5\" (UID: \"04be873e-1f09-4680-be2a-5d7f3e7d5235\") " pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:55:08.813965 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.813938    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:55:08.839751 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.839719    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6479bbd56d-kgdjl"]
Oct 30 18:55:08.846682 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:55:08.846646    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01509dac_efd0_49cc_b923_a5c1835623e2.slice/crio-391200e0da55837f761ccc9fd11cb5cdbb5e156e60b2de477e728d01e5c1581c WatchSource:0}: Error finding container 391200e0da55837f761ccc9fd11cb5cdbb5e156e60b2de477e728d01e5c1581c: Status 404 returned error can't find the container with id 391200e0da55837f761ccc9fd11cb5cdbb5e156e60b2de477e728d01e5c1581c
Oct 30 18:55:08.921269 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.921170    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt" event={"ID":"ad673c6d-2077-4cdc-8628-4fda925c1517","Type":"ContainerStarted","Data":"fee3693c453e256ee3aee4b5eef567eba821437f1d0ff79e8837feaac7f45b47"}
Oct 30 18:55:08.922555 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:08.922529    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6479bbd56d-kgdjl" event={"ID":"01509dac-efd0-49cc-b923-a5c1835623e2","Type":"ContainerStarted","Data":"391200e0da55837f761ccc9fd11cb5cdbb5e156e60b2de477e728d01e5c1581c"}
Oct 30 18:55:09.187625 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:09.187487    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8558845494-2hrn5"]
Oct 30 18:55:09.193059 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:55:09.193024    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04be873e_1f09_4680_be2a_5d7f3e7d5235.slice/crio-ce413e07fa8bce53545f96a28983e22eb07a34da24900a7edd321bf51f45b40b WatchSource:0}: Error finding container ce413e07fa8bce53545f96a28983e22eb07a34da24900a7edd321bf51f45b40b: Status 404 returned error can't find the container with id ce413e07fa8bce53545f96a28983e22eb07a34da24900a7edd321bf51f45b40b
Oct 30 18:55:09.928253 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:09.928217    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8558845494-2hrn5" event={"ID":"04be873e-1f09-4680-be2a-5d7f3e7d5235","Type":"ContainerStarted","Data":"ce413e07fa8bce53545f96a28983e22eb07a34da24900a7edd321bf51f45b40b"}
Oct 30 18:55:10.944118 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:10.943431    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6479bbd56d-kgdjl" event={"ID":"01509dac-efd0-49cc-b923-a5c1835623e2","Type":"ContainerStarted","Data":"b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53"}
Oct 30 18:55:10.944118 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:10.943612    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6479bbd56d-kgdjl" podUID="01509dac-efd0-49cc-b923-a5c1835623e2" containerName="authorino" containerID="cri-o://b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53" gracePeriod=30
Oct 30 18:55:10.958984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:10.958944    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt" event={"ID":"ad673c6d-2077-4cdc-8628-4fda925c1517","Type":"ContainerStarted","Data":"f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a"}
Oct 30 18:55:10.959142 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:10.959059    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt" podUID="ad673c6d-2077-4cdc-8628-4fda925c1517" containerName="authorino" containerID="cri-o://f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a" gracePeriod=30
Oct 30 18:55:10.961610 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:10.961560    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8558845494-2hrn5" event={"ID":"04be873e-1f09-4680-be2a-5d7f3e7d5235","Type":"ContainerStarted","Data":"493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58"}
Oct 30 18:55:11.060394 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.060343    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6479bbd56d-kgdjl" podStartSLOduration=1.6275770440000001 podStartE2EDuration="3.060327598s" totalImagesPullingTime="1.432750554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:11.060033675 +0000 UTC m=+503.644516673" watchObservedRunningTime="2026-10-30 18:55:11.060327598 +0000 UTC m=+503.644810589"
Oct 30 18:55:11.174310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.172904    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-8558845494-2hrn5" podStartSLOduration=1.846120387 podStartE2EDuration="3.172889409s" totalImagesPullingTime="1.326769022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:11.17206378 +0000 UTC m=+503.756546776" watchObservedRunningTime="2026-10-30 18:55:11.172889409 +0000 UTC m=+503.757372406"
Oct 30 18:55:11.206850 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.206739    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-mgd48"]
Oct 30 18:55:11.207132 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.206996    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48" podUID="8b0cf4b1-708b-484d-8497-d00966e3e270" containerName="authorino" containerID="cri-o://6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d" gracePeriod=30
Oct 30 18:55:11.290696 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.289933    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt" podStartSLOduration=2.596667489 podStartE2EDuration="4.289913175s" totalImagesPullingTime="1.693245686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:11.287002934 +0000 UTC m=+503.871485932" watchObservedRunningTime="2026-10-30 18:55:11.289913175 +0000 UTC m=+503.874396174"
Oct 30 18:55:11.348448 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.348413    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6479bbd56d-kgdjl"
Oct 30 18:55:11.353125 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.353100    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt"
Oct 30 18:55:11.472595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.471835    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01509dac-efd0-49cc-b923-a5c1835623e2-kube-api-access-59wj9\" (UniqueName: \"kubernetes.io/projected/01509dac-efd0-49cc-b923-a5c1835623e2-kube-api-access-59wj9\") pod \"01509dac-efd0-49cc-b923-a5c1835623e2\" (UID: \"01509dac-efd0-49cc-b923-a5c1835623e2\") "
Oct 30 18:55:11.472595 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.471969    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad673c6d-2077-4cdc-8628-4fda925c1517-kube-api-access-lmcn9\" (UniqueName: \"kubernetes.io/projected/ad673c6d-2077-4cdc-8628-4fda925c1517-kube-api-access-lmcn9\") pod \"ad673c6d-2077-4cdc-8628-4fda925c1517\" (UID: \"ad673c6d-2077-4cdc-8628-4fda925c1517\") "
Oct 30 18:55:11.473943 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.473919    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48"
Oct 30 18:55:11.475444 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.475412    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01509dac-efd0-49cc-b923-a5c1835623e2-kube-api-access-59wj9" pod "01509dac-efd0-49cc-b923-a5c1835623e2" (UID: "01509dac-efd0-49cc-b923-a5c1835623e2"). InnerVolumeSpecName "kube-api-access-59wj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:11.479967 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.479937    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad673c6d-2077-4cdc-8628-4fda925c1517-kube-api-access-lmcn9" pod "ad673c6d-2077-4cdc-8628-4fda925c1517" (UID: "ad673c6d-2077-4cdc-8628-4fda925c1517"). InnerVolumeSpecName "kube-api-access-lmcn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:11.573018 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.572984    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b0cf4b1-708b-484d-8497-d00966e3e270-kube-api-access-b8qdt\" (UniqueName: \"kubernetes.io/projected/8b0cf4b1-708b-484d-8497-d00966e3e270-kube-api-access-b8qdt\") pod \"8b0cf4b1-708b-484d-8497-d00966e3e270\" (UID: \"8b0cf4b1-708b-484d-8497-d00966e3e270\") "
Oct 30 18:55:11.573283 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.573265    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59wj9\" (UniqueName: \"kubernetes.io/projected/01509dac-efd0-49cc-b923-a5c1835623e2-kube-api-access-59wj9\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:11.573326 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.573290    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmcn9\" (UniqueName: \"kubernetes.io/projected/ad673c6d-2077-4cdc-8628-4fda925c1517-kube-api-access-lmcn9\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:11.575135 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.575107    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b0cf4b1-708b-484d-8497-d00966e3e270-kube-api-access-b8qdt" pod "8b0cf4b1-708b-484d-8497-d00966e3e270" (UID: "8b0cf4b1-708b-484d-8497-d00966e3e270"). InnerVolumeSpecName "kube-api-access-b8qdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:11.674358 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.674321    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8qdt\" (UniqueName: \"kubernetes.io/projected/8b0cf4b1-708b-484d-8497-d00966e3e270-kube-api-access-b8qdt\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:11.968473 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.968440    5232 generic.go:356] "Generic (PLEG): container finished" podID="8b0cf4b1-708b-484d-8497-d00966e3e270" containerID="6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d" exitCode=0
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.968497    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48"
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.968492    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48" event={"ID":"8b0cf4b1-708b-484d-8497-d00966e3e270","Type":"ContainerDied","Data":"6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d"}
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.968629    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-mgd48" event={"ID":"8b0cf4b1-708b-484d-8497-d00966e3e270","Type":"ContainerDied","Data":"15d518020f1e94997cf612f04c869f1507231954321d32cfc3c0a062385c12db"}
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.968654    5232 scope.go:122] "RemoveContainer" containerID="6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d"
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.969903    5232 generic.go:356] "Generic (PLEG): container finished" podID="01509dac-efd0-49cc-b923-a5c1835623e2" containerID="b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53" exitCode=0
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.969956    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6479bbd56d-kgdjl"
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.969958    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6479bbd56d-kgdjl" event={"ID":"01509dac-efd0-49cc-b923-a5c1835623e2","Type":"ContainerDied","Data":"b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53"}
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.969989    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6479bbd56d-kgdjl" event={"ID":"01509dac-efd0-49cc-b923-a5c1835623e2","Type":"ContainerDied","Data":"391200e0da55837f761ccc9fd11cb5cdbb5e156e60b2de477e728d01e5c1581c"}
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.971489    5232 generic.go:356] "Generic (PLEG): container finished" podID="ad673c6d-2077-4cdc-8628-4fda925c1517" containerID="f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a" exitCode=0
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.971566    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt" event={"ID":"ad673c6d-2077-4cdc-8628-4fda925c1517","Type":"ContainerDied","Data":"f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a"}
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.971601    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt" event={"ID":"ad673c6d-2077-4cdc-8628-4fda925c1517","Type":"ContainerDied","Data":"fee3693c453e256ee3aee4b5eef567eba821437f1d0ff79e8837feaac7f45b47"}
Oct 30 18:55:11.978928 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.971632    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-qcfqt"
Oct 30 18:55:11.979782 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.979159    5232 scope.go:122] "RemoveContainer" containerID="6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d"
Oct 30 18:55:11.979782 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:55:11.979462    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d\": container with ID starting with 6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d not found: ID does not exist" containerID="6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d"
Oct 30 18:55:11.979782 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.979488    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d"} err="failed to get container status \"6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d\": rpc error: code = NotFound desc = could not find container \"6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d\": container with ID starting with 6b6e2c464a077716c2670f5ded3d5deec76ebc1e0cb280a79c48e0573d5c3e0d not found: ID does not exist"
Oct 30 18:55:11.979782 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.979512    5232 scope.go:122] "RemoveContainer" containerID="b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53"
Oct 30 18:55:11.987310 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.987290    5232 scope.go:122] "RemoveContainer" containerID="b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53"
Oct 30 18:55:11.987570 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:55:11.987551    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53\": container with ID starting with b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53 not found: ID does not exist" containerID="b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53"
Oct 30 18:55:11.987617 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.987579    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53"} err="failed to get container status \"b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53\": rpc error: code = NotFound desc = could not find container \"b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53\": container with ID starting with b6d323b24d90dd7773532a37a3ec8427533f4295beb8f7949cea535fe6b98d53 not found: ID does not exist"
Oct 30 18:55:11.987617 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.987595    5232 scope.go:122] "RemoveContainer" containerID="f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a"
Oct 30 18:55:11.995297 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.995280    5232 scope.go:122] "RemoveContainer" containerID="f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a"
Oct 30 18:55:11.995556 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:55:11.995537    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a\": container with ID starting with f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a not found: ID does not exist" containerID="f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a"
Oct 30 18:55:11.995611 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:11.995564    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a"} err="failed to get container status \"f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a\": rpc error: code = NotFound desc = could not find container \"f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a\": container with ID starting with f51bf133aab0630ca28ef194bdd42b4267678c9219af42b8edee26e38439683a not found: ID does not exist"
Oct 30 18:55:12.126400 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.126368    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-mgd48"]
Oct 30 18:55:12.128558 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.128535    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-mgd48"]
Oct 30 18:55:12.233427 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.233329    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6479bbd56d-kgdjl"]
Oct 30 18:55:12.237843 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.237799    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6479bbd56d-kgdjl"]
Oct 30 18:55:12.342181 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.342150    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-qcfqt"]
Oct 30 18:55:12.347230 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.347202    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-qcfqt"]
Oct 30 18:55:12.979982 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.979943    5232 generic.go:356] "Generic (PLEG): container finished" podID="ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a" containerID="a3a77426a8db554d2e408b6179ccfd8f6192b50b57099830c72fc99897e66acb" exitCode=0
Oct 30 18:55:12.980403 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.980020    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" event={"ID":"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a","Type":"ContainerDied","Data":"a3a77426a8db554d2e408b6179ccfd8f6192b50b57099830c72fc99897e66acb"}
Oct 30 18:55:12.980488 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.980452    5232 scope.go:122] "RemoveContainer" containerID="a3a77426a8db554d2e408b6179ccfd8f6192b50b57099830c72fc99897e66acb"
Oct 30 18:55:12.981703 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.981671    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:55:12.981923 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.981756    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:55:12.981923 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:12.981808    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:55:13.548895 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:13.538985    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01509dac-efd0-49cc-b923-a5c1835623e2" path="/var/lib/kubelet/pods/01509dac-efd0-49cc-b923-a5c1835623e2/volumes"
Oct 30 18:55:13.548895 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:13.539440    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b0cf4b1-708b-484d-8497-d00966e3e270" path="/var/lib/kubelet/pods/8b0cf4b1-708b-484d-8497-d00966e3e270/volumes"
Oct 30 18:55:13.561852 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:13.558997    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad673c6d-2077-4cdc-8628-4fda925c1517" path="/var/lib/kubelet/pods/ad673c6d-2077-4cdc-8628-4fda925c1517/volumes"
Oct 30 18:55:13.989587 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:13.989555    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" event={"ID":"ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a","Type":"ContainerStarted","Data":"25eb5219df1280de4070fad7834162cb0b749aff7772ea57c34d9d16c160db20"}
Oct 30 18:55:14.953463 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:14.953428    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:55:14.953830 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:14.953785    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-gshcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused" start-of-body=
Oct 30 18:55:14.954010 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:14.953981    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" podUID="ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused"
Oct 30 18:55:15.813540 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:15.813504    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-6fh88"
Oct 30 18:55:15.947241 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:15.947205    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-gshcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused" start-of-body=
Oct 30 18:55:15.947493 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:15.947464    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" podUID="ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused"
Oct 30 18:55:16.188071 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:55:16.188037    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:55:16.955837 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:16.954179    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-gshcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused" start-of-body=
Oct 30 18:55:16.955837 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:16.954242    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" podUID="ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused"
Oct 30 18:55:17.952547 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:17.952210    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-gshcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused" start-of-body=
Oct 30 18:55:17.952547 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:17.952274    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" podUID="ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused"
Oct 30 18:55:18.264242 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.264143    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"]
Oct 30 18:55:18.264774 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.264401    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5" podUID="72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" containerName="maas-api" containerID="cri-o://ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0" gracePeriod=30
Oct 30 18:55:18.512847 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.512822    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:55:18.671478 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.670977    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-maas-api-tls\") pod \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") "
Oct 30 18:55:18.671478 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.671044    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-openshift-service-ca\") pod \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") "
Oct 30 18:55:18.671478 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.671103    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-kube-api-access-26hqk\" (UniqueName: \"kubernetes.io/projected/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-kube-api-access-26hqk\") pod \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") "
Oct 30 18:55:18.671478 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.671128    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-metrics-certs\") pod \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\" (UID: \"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5\") "
Oct 30 18:55:18.671796 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.671475    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-openshift-service-ca" pod "72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" (UID: "72dc80b9-c8d3-4cbf-8e98-a17fc64097c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:55:18.673251 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.673222    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-maas-api-tls" pod "72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" (UID: "72dc80b9-c8d3-4cbf-8e98-a17fc64097c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:55:18.673800 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.673777    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-metrics-certs" pod "72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" (UID: "72dc80b9-c8d3-4cbf-8e98-a17fc64097c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:55:18.676939 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.676912    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-kube-api-access-26hqk" pod "72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" (UID: "72dc80b9-c8d3-4cbf-8e98-a17fc64097c5"). InnerVolumeSpecName "kube-api-access-26hqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:18.772059 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.772023    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:18.772059 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.772054    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26hqk\" (UniqueName: \"kubernetes.io/projected/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-kube-api-access-26hqk\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:18.772059 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.772065    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:18.772294 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.772075    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:18.947090 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.947010    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-gshcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused" start-of-body=
Oct 30 18:55:18.947090 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:18.947060    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc" podUID="ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.21:15021/healthz/ready\": dial tcp 10.133.0.21:15021: connect: connection refused"
Oct 30 18:55:19.016712 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.016671    5232 generic.go:356] "Generic (PLEG): container finished" podID="72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" containerID="ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0" exitCode=0
Oct 30 18:55:19.016900 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.016733    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"
Oct 30 18:55:19.016900 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.016755    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5" event={"ID":"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5","Type":"ContainerDied","Data":"ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0"}
Oct 30 18:55:19.016900 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.016825    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5" event={"ID":"72dc80b9-c8d3-4cbf-8e98-a17fc64097c5","Type":"ContainerDied","Data":"67b017e5cf8cf02d636f708af739ec4cd1f130b1665f66e0614a18fc2de44e8b"}
Oct 30 18:55:19.016900 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.016852    5232 scope.go:122] "RemoveContainer" containerID="ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0"
Oct 30 18:55:19.025963 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.025900    5232 scope.go:122] "RemoveContainer" containerID="ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0"
Oct 30 18:55:19.026230 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:55:19.026199    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0\": container with ID starting with ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0 not found: ID does not exist" containerID="ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0"
Oct 30 18:55:19.026343 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.026231    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0"} err="failed to get container status \"ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0\": rpc error: code = NotFound desc = could not find container \"ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0\": container with ID starting with ac09f33283d91e66d25370622db5543e1d82acf668087f05489371cd459824e0 not found: ID does not exist"
Oct 30 18:55:19.175429 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.175395    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"]
Oct 30 18:55:19.181093 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.181064    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-764f864bb9-rwxr5"]
Oct 30 18:55:19.537977 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.537945    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" path="/var/lib/kubelet/pods/72dc80b9-c8d3-4cbf-8e98-a17fc64097c5/volumes"
Oct 30 18:55:19.951300 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.951269    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:55:19.951670 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.951651    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:55:19.952326 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:55:19.952305    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-gshcc"
Oct 30 18:55:46.186669 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:55:46.186627    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:56:16.186159 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:56:16.186124    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:56:27.035317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035285    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-hx6dl"]
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035594    5232 cpu_manager.go:414] "containerMap: removing container" podUID="8b0cf4b1-708b-484d-8497-d00966e3e270" containerName="authorino"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035604    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b0cf4b1-708b-484d-8497-d00966e3e270" containerName="authorino"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035615    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ad673c6d-2077-4cdc-8628-4fda925c1517" containerName="authorino"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035623    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad673c6d-2077-4cdc-8628-4fda925c1517" containerName="authorino"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035656    5232 cpu_manager.go:414] "containerMap: removing container" podUID="72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" containerName="maas-api"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035661    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" containerName="maas-api"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035669    5232 cpu_manager.go:414] "containerMap: removing container" podUID="01509dac-efd0-49cc-b923-a5c1835623e2" containerName="authorino"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035674    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01509dac-efd0-49cc-b923-a5c1835623e2" containerName="authorino"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035722    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b0cf4b1-708b-484d-8497-d00966e3e270" containerName="authorino"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035729    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad673c6d-2077-4cdc-8628-4fda925c1517" containerName="authorino"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035747    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="72dc80b9-c8d3-4cbf-8e98-a17fc64097c5" containerName="maas-api"
Oct 30 18:56:27.035760 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.035753    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="01509dac-efd0-49cc-b923-a5c1835623e2" containerName="authorino"
Oct 30 18:56:27.039165 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.039143    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:27.138935 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.138899    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 18:56:27.138935 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.138923    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-r7zq9\""
Oct 30 18:56:27.139131 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.138982    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 18:56:27.140770 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.140753    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-hx6dl"]
Oct 30 18:56:27.149938 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.149912    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcswd\" (UniqueName: \"kubernetes.io/projected/5ea3a907-0d98-4223-9cb1-023207e21e42-kube-api-access-vcswd\") pod \"ai-gateway-controller-7d85675f98-hx6dl\" (UID: \"5ea3a907-0d98-4223-9cb1-023207e21e42\") " pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:27.150036 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.149971    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ea3a907-0d98-4223-9cb1-023207e21e42-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-hx6dl\" (UID: \"5ea3a907-0d98-4223-9cb1-023207e21e42\") " pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:27.250663 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.250626    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcswd\" (UniqueName: \"kubernetes.io/projected/5ea3a907-0d98-4223-9cb1-023207e21e42-kube-api-access-vcswd\") pod \"ai-gateway-controller-7d85675f98-hx6dl\" (UID: \"5ea3a907-0d98-4223-9cb1-023207e21e42\") " pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:27.250860 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.250707    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ea3a907-0d98-4223-9cb1-023207e21e42-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-hx6dl\" (UID: \"5ea3a907-0d98-4223-9cb1-023207e21e42\") " pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:27.253010 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.252988    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ea3a907-0d98-4223-9cb1-023207e21e42-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-hx6dl\" (UID: \"5ea3a907-0d98-4223-9cb1-023207e21e42\") " pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:27.311430 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.311360    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcswd\" (UniqueName: \"kubernetes.io/projected/5ea3a907-0d98-4223-9cb1-023207e21e42-kube-api-access-vcswd\") pod \"ai-gateway-controller-7d85675f98-hx6dl\" (UID: \"5ea3a907-0d98-4223-9cb1-023207e21e42\") " pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:27.352103 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.352073    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:27.720244 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:27.720213    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-hx6dl"]
Oct 30 18:56:27.726066 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:56:27.726037    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea3a907_0d98_4223_9cb1_023207e21e42.slice/crio-a8d49d1b663c7f263ee989eced15bc4451bdaf9e4ee78b2f5e3ee5d7fe65fcab WatchSource:0}: Error finding container a8d49d1b663c7f263ee989eced15bc4451bdaf9e4ee78b2f5e3ee5d7fe65fcab: Status 404 returned error can't find the container with id a8d49d1b663c7f263ee989eced15bc4451bdaf9e4ee78b2f5e3ee5d7fe65fcab
Oct 30 18:56:28.353903 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:28.353857    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl" event={"ID":"5ea3a907-0d98-4223-9cb1-023207e21e42","Type":"ContainerStarted","Data":"a8d49d1b663c7f263ee989eced15bc4451bdaf9e4ee78b2f5e3ee5d7fe65fcab"}
Oct 30 18:56:33.380460 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:33.380422    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl" event={"ID":"5ea3a907-0d98-4223-9cb1-023207e21e42","Type":"ContainerStarted","Data":"3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce"}
Oct 30 18:56:33.380909 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:33.380557    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:33.493084 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:33.493034    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl" podStartSLOduration=2.64406335 podStartE2EDuration="7.493020784s" totalImagesPullingTime="4.848957434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:33.492657688 +0000 UTC m=+586.077140686" watchObservedRunningTime="2026-10-30 18:56:33.493020784 +0000 UTC m=+586.077503780"
Oct 30 18:56:36.820834 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.820778    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"]
Oct 30 18:56:36.824226 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.824204    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:36.921888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.921854    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 18:56:36.921888 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.921858    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7jtjf\""
Oct 30 18:56:36.925138 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.925106    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"]
Oct 30 18:56:36.938268 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.938234    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f035666f-23c9-494a-9963-6795a9f87e59-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-v65bt\" (UID: \"f035666f-23c9-494a-9963-6795a9f87e59\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:36.938571 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.938543    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrnqx\" (UniqueName: \"kubernetes.io/projected/f035666f-23c9-494a-9963-6795a9f87e59-kube-api-access-qrnqx\") pod \"payload-pre-processing-7b98bb678c-v65bt\" (UID: \"f035666f-23c9-494a-9963-6795a9f87e59\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:36.990257 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.990223    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-9zk2f"]
Oct 30 18:56:36.993579 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:36.993554    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:37.039967 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.039916    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/528babed-84aa-491f-85a9-16cc40b8edc9-plugins-config-volume\") pod \"payload-processing-5f78687d5b-9zk2f\" (UID: \"528babed-84aa-491f-85a9-16cc40b8edc9\") " pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:37.040166 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.040087    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f035666f-23c9-494a-9963-6795a9f87e59-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-v65bt\" (UID: \"f035666f-23c9-494a-9963-6795a9f87e59\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:37.040166 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.040138    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b96gl\" (UniqueName: \"kubernetes.io/projected/528babed-84aa-491f-85a9-16cc40b8edc9-kube-api-access-b96gl\") pod \"payload-processing-5f78687d5b-9zk2f\" (UID: \"528babed-84aa-491f-85a9-16cc40b8edc9\") " pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:37.040281 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.040197    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrnqx\" (UniqueName: \"kubernetes.io/projected/f035666f-23c9-494a-9963-6795a9f87e59-kube-api-access-qrnqx\") pod \"payload-pre-processing-7b98bb678c-v65bt\" (UID: \"f035666f-23c9-494a-9963-6795a9f87e59\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:37.041100 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.041077    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f035666f-23c9-494a-9963-6795a9f87e59-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-v65bt\" (UID: \"f035666f-23c9-494a-9963-6795a9f87e59\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:37.054290 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.054257    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-9zk2f"]
Oct 30 18:56:37.099303 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.099276    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrnqx\" (UniqueName: \"kubernetes.io/projected/f035666f-23c9-494a-9963-6795a9f87e59-kube-api-access-qrnqx\") pod \"payload-pre-processing-7b98bb678c-v65bt\" (UID: \"f035666f-23c9-494a-9963-6795a9f87e59\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:37.136275 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.136249    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:37.141432 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.141410    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/528babed-84aa-491f-85a9-16cc40b8edc9-plugins-config-volume\") pod \"payload-processing-5f78687d5b-9zk2f\" (UID: \"528babed-84aa-491f-85a9-16cc40b8edc9\") " pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:37.141548 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.141492    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b96gl\" (UniqueName: \"kubernetes.io/projected/528babed-84aa-491f-85a9-16cc40b8edc9-kube-api-access-b96gl\") pod \"payload-processing-5f78687d5b-9zk2f\" (UID: \"528babed-84aa-491f-85a9-16cc40b8edc9\") " pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:37.142071 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.142051    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/528babed-84aa-491f-85a9-16cc40b8edc9-plugins-config-volume\") pod \"payload-processing-5f78687d5b-9zk2f\" (UID: \"528babed-84aa-491f-85a9-16cc40b8edc9\") " pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:37.204448 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.204414    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b96gl\" (UniqueName: \"kubernetes.io/projected/528babed-84aa-491f-85a9-16cc40b8edc9-kube-api-access-b96gl\") pod \"payload-processing-5f78687d5b-9zk2f\" (UID: \"528babed-84aa-491f-85a9-16cc40b8edc9\") " pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:37.306867 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.306835    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:37.505557 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.505517    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"]
Oct 30 18:56:37.511434 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:56:37.511410    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf035666f_23c9_494a_9963_6795a9f87e59.slice/crio-5d11e6ca8c76fbce40e319024263519de750225c37e6a8f6d5ee6e3cae962c65 WatchSource:0}: Error finding container 5d11e6ca8c76fbce40e319024263519de750225c37e6a8f6d5ee6e3cae962c65: Status 404 returned error can't find the container with id 5d11e6ca8c76fbce40e319024263519de750225c37e6a8f6d5ee6e3cae962c65
Oct 30 18:56:37.676302 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:37.676223    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-9zk2f"]
Oct 30 18:56:37.682118 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:56:37.682090    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod528babed_84aa_491f_85a9_16cc40b8edc9.slice/crio-3bc121057b7123355fafee3604b95479e35425fea2091d585d4a97b90ab0d491 WatchSource:0}: Error finding container 3bc121057b7123355fafee3604b95479e35425fea2091d585d4a97b90ab0d491: Status 404 returned error can't find the container with id 3bc121057b7123355fafee3604b95479e35425fea2091d585d4a97b90ab0d491
Oct 30 18:56:38.406017 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:38.405982    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt" event={"ID":"f035666f-23c9-494a-9963-6795a9f87e59","Type":"ContainerStarted","Data":"5d11e6ca8c76fbce40e319024263519de750225c37e6a8f6d5ee6e3cae962c65"}
Oct 30 18:56:38.407202 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:38.407177    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f" event={"ID":"528babed-84aa-491f-85a9-16cc40b8edc9","Type":"ContainerStarted","Data":"3bc121057b7123355fafee3604b95479e35425fea2091d585d4a97b90ab0d491"}
Oct 30 18:56:40.421254 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:40.421223    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt" event={"ID":"f035666f-23c9-494a-9963-6795a9f87e59","Type":"ContainerStarted","Data":"22d28618bd6ab715b3577f08f6f6ca74d8bd8736f0c5f52f68bccba1cd3364c2"}
Oct 30 18:56:40.421637 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:40.421386    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:40.422926 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:40.422900    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f" event={"ID":"528babed-84aa-491f-85a9-16cc40b8edc9","Type":"ContainerStarted","Data":"2520cd490aa7bff712560f06c0e02e8c8d927fbb02a7f852b51716bfeec4c989"}
Oct 30 18:56:40.423052 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:40.423036    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:40.538073 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:40.538011    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt" podStartSLOduration=1.713163905 podStartE2EDuration="4.537993488s" totalImagesPullingTime="2.824829583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:40.536286461 +0000 UTC m=+593.120769458" watchObservedRunningTime="2026-10-30 18:56:40.537993488 +0000 UTC m=+593.122476489"
Oct 30 18:56:40.650837 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:40.650766    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f" podStartSLOduration=2.005719652 podStartE2EDuration="4.650752392s" totalImagesPullingTime="2.64503274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:40.649897123 +0000 UTC m=+593.234380121" watchObservedRunningTime="2026-10-30 18:56:40.650752392 +0000 UTC m=+593.235235388"
Oct 30 18:56:44.387449 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:44.387418    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:56:46.186411 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:56:46.186374    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:56:47.565868 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:47.565839    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 18:56:47.566712 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:47.566690    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 18:56:47.570091 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:47.570070    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 18:56:47.571078 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:47.571061    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 18:56:51.430845 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:51.430805    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-9zk2f"
Oct 30 18:56:51.431227 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:51.430970    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-v65bt"
Oct 30 18:56:52.481049 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:52.481016    5232 generic.go:356] "Generic (PLEG): container finished" podID="9070de12-c761-46e4-a362-2365219f92eb" containerID="2f0bff07b5d09ef6e560949a4c5cd69fa8f20f0ffb290d667b85bb87a614973a" exitCode=1
Oct 30 18:56:52.481464 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:52.481092    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7447664ff8-lfrzx" event={"ID":"9070de12-c761-46e4-a362-2365219f92eb","Type":"ContainerDied","Data":"2f0bff07b5d09ef6e560949a4c5cd69fa8f20f0ffb290d667b85bb87a614973a"}
Oct 30 18:56:52.481539 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:52.481474    5232 scope.go:122] "RemoveContainer" containerID="2f0bff07b5d09ef6e560949a4c5cd69fa8f20f0ffb290d667b85bb87a614973a"
Oct 30 18:56:52.481867 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:52.481853    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:56:53.488367 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:53.488329    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7447664ff8-lfrzx" event={"ID":"9070de12-c761-46e4-a362-2365219f92eb","Type":"ContainerStarted","Data":"9d1fa777f2b3d74b1b95af291ab6db31cce175fa8afda42f4b09f9a89d4176c7"}
Oct 30 18:56:53.488749 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:53.488570    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:56:59.665623 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:59.665592    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-hx6dl"]
Oct 30 18:56:59.666092 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:59.665765    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl" podUID="5ea3a907-0d98-4223-9cb1-023207e21e42" containerName="manager" containerID="cri-o://3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce" gracePeriod=30
Oct 30 18:56:59.913421 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:56:59.913390    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:57:00.050109 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.050030    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ea3a907-0d98-4223-9cb1-023207e21e42-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ea3a907-0d98-4223-9cb1-023207e21e42-metrics-certs\") pod \"5ea3a907-0d98-4223-9cb1-023207e21e42\" (UID: \"5ea3a907-0d98-4223-9cb1-023207e21e42\") "
Oct 30 18:57:00.050109 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.050082    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ea3a907-0d98-4223-9cb1-023207e21e42-kube-api-access-vcswd\" (UniqueName: \"kubernetes.io/projected/5ea3a907-0d98-4223-9cb1-023207e21e42-kube-api-access-vcswd\") pod \"5ea3a907-0d98-4223-9cb1-023207e21e42\" (UID: \"5ea3a907-0d98-4223-9cb1-023207e21e42\") "
Oct 30 18:57:00.052271 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.052244    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea3a907-0d98-4223-9cb1-023207e21e42-metrics-certs" pod "5ea3a907-0d98-4223-9cb1-023207e21e42" (UID: "5ea3a907-0d98-4223-9cb1-023207e21e42"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:00.052271 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.052251    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea3a907-0d98-4223-9cb1-023207e21e42-kube-api-access-vcswd" pod "5ea3a907-0d98-4223-9cb1-023207e21e42" (UID: "5ea3a907-0d98-4223-9cb1-023207e21e42"). InnerVolumeSpecName "kube-api-access-vcswd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:00.151441 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.151404    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ea3a907-0d98-4223-9cb1-023207e21e42-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:00.151441 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.151439    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcswd\" (UniqueName: \"kubernetes.io/projected/5ea3a907-0d98-4223-9cb1-023207e21e42-kube-api-access-vcswd\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:00.522298 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.522264    5232 generic.go:356] "Generic (PLEG): container finished" podID="5ea3a907-0d98-4223-9cb1-023207e21e42" containerID="3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce" exitCode=0
Oct 30 18:57:00.522499 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.522324    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl"
Oct 30 18:57:00.522499 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.522351    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl" event={"ID":"5ea3a907-0d98-4223-9cb1-023207e21e42","Type":"ContainerDied","Data":"3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce"}
Oct 30 18:57:00.522499 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.522414    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-hx6dl" event={"ID":"5ea3a907-0d98-4223-9cb1-023207e21e42","Type":"ContainerDied","Data":"a8d49d1b663c7f263ee989eced15bc4451bdaf9e4ee78b2f5e3ee5d7fe65fcab"}
Oct 30 18:57:00.522499 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.522443    5232 scope.go:122] "RemoveContainer" containerID="3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce"
Oct 30 18:57:00.532284 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.532262    5232 scope.go:122] "RemoveContainer" containerID="3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce"
Oct 30 18:57:00.532596 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:57:00.532571    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce\": container with ID starting with 3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce not found: ID does not exist" containerID="3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce"
Oct 30 18:57:00.532696 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.532607    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce"} err="failed to get container status \"3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce\": rpc error: code = NotFound desc = could not find container \"3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce\": container with ID starting with 3bb750597592bf0d62f185f665646256cb7a513bcbc32324ec6b1f4c7eca46ce not found: ID does not exist"
Oct 30 18:57:00.680521 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.680488    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-hx6dl"]
Oct 30 18:57:00.728200 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:00.728160    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-hx6dl"]
Oct 30 18:57:01.538165 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:01.538133    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ea3a907-0d98-4223-9cb1-023207e21e42" path="/var/lib/kubelet/pods/5ea3a907-0d98-4223-9cb1-023207e21e42/volumes"
Oct 30 18:57:04.494895 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:04.494865    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7447664ff8-lfrzx"
Oct 30 18:57:16.188032 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:57:16.187990    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:57:20.627719 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.627685    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"]
Oct 30 18:57:20.628193 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.628133    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5ea3a907-0d98-4223-9cb1-023207e21e42" containerName="manager"
Oct 30 18:57:20.628193 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.628149    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ea3a907-0d98-4223-9cb1-023207e21e42" containerName="manager"
Oct 30 18:57:20.628275 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.628234    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ea3a907-0d98-4223-9cb1-023207e21e42" containerName="manager"
Oct 30 18:57:20.634745 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.634723    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.686508 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.686482    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 18:57:20.717295 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.717253    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.717490 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.717316    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.717490 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.717362    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.717490 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.717387    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.717980 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.717490    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8rpq\" (UniqueName: \"kubernetes.io/projected/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-kube-api-access-l8rpq\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.717980 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.717519    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.734088 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.734055    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b4g6l\""
Oct 30 18:57:20.734207 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.734172    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 18:57:20.734282 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.734265    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 18:57:20.736187 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.736166    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"]
Oct 30 18:57:20.818575 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.818539    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.818575 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.818581    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.818785 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.818629    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8rpq\" (UniqueName: \"kubernetes.io/projected/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-kube-api-access-l8rpq\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.818785 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.818649    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.818785 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.818677    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.818785 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.818727    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.819043 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.819019    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.819103 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.819072    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.819103 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.819092    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.820844 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.820801    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.821180 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.821163    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.878101 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.878024    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8rpq\" (UniqueName: \"kubernetes.io/projected/3a777aa3-f14b-4ae3-b229-1b80b9ec54eb-kube-api-access-l8rpq\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4\" (UID: \"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:20.947029 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:20.946993    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:21.320067 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:21.319976    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"]
Oct 30 18:57:21.325985 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:57:21.325955    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a777aa3_f14b_4ae3_b229_1b80b9ec54eb.slice/crio-98ce9591110cf190ed20fa20fe5be02b2d3aa96f06100fa585a7785112c8c5a1 WatchSource:0}: Error finding container 98ce9591110cf190ed20fa20fe5be02b2d3aa96f06100fa585a7785112c8c5a1: Status 404 returned error can't find the container with id 98ce9591110cf190ed20fa20fe5be02b2d3aa96f06100fa585a7785112c8c5a1
Oct 30 18:57:21.624563 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:21.624529    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4" event={"ID":"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb","Type":"ContainerStarted","Data":"98ce9591110cf190ed20fa20fe5be02b2d3aa96f06100fa585a7785112c8c5a1"}
Oct 30 18:57:23.636118 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:23.636078    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4" event={"ID":"3a777aa3-f14b-4ae3-b229-1b80b9ec54eb","Type":"ContainerStarted","Data":"b4ea77a80914523d23909ed04bf8b174c4dbd0b6421fba42884960d65fdccf45"}
Oct 30 18:57:23.751041 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:23.750984    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4" podStartSLOduration=1.546633817 podStartE2EDuration="3.750968671s" totalImagesPullingTime="2.204334854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:23.750035219 +0000 UTC m=+636.334518215" watchObservedRunningTime="2026-10-30 18:57:23.750968671 +0000 UTC m=+636.335451669"
Oct 30 18:57:23.947577 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:23.947537    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:30.879125 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.879040    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"]
Oct 30 18:57:30.882712 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.882691    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:30.910600 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.910575    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:30.910719 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.910615    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:30.910719 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.910649    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c31cd17-e19d-43a0-9a77-f6819eae43fb-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:30.910788 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.910732    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:30.910788 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.910769    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbl7x\" (UniqueName: \"kubernetes.io/projected/4c31cd17-e19d-43a0-9a77-f6819eae43fb-kube-api-access-dbl7x\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:30.910886 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.910790    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:30.939962 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.939926    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 18:57:30.948584 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.947823    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:30.948768 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.948600    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"]
Oct 30 18:57:30.964554 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:30.964521    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:31.012157 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.012127    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.012360 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.012178    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbl7x\" (UniqueName: \"kubernetes.io/projected/4c31cd17-e19d-43a0-9a77-f6819eae43fb-kube-api-access-dbl7x\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.012360 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.012201    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.012360 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.012234    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.012360 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.012302    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.012360 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.012357    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c31cd17-e19d-43a0-9a77-f6819eae43fb-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.013059 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.012679    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.013553 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.013524    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.013666 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.013594    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.024843 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.023029    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c31cd17-e19d-43a0-9a77-f6819eae43fb-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.024843 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.023259    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c31cd17-e19d-43a0-9a77-f6819eae43fb-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.070976 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.070945    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbl7x\" (UniqueName: \"kubernetes.io/projected/4c31cd17-e19d-43a0-9a77-f6819eae43fb-kube-api-access-dbl7x\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wtpr6\" (UID: \"4c31cd17-e19d-43a0-9a77-f6819eae43fb\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.201646 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.201560    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:31.574762 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.574683    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"]
Oct 30 18:57:31.580201 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:57:31.580176    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c31cd17_e19d_43a0_9a77_f6819eae43fb.slice/crio-fd10218e822b7d84dd08da69e3d78d002adf0e2c7e5e7491f4b7b38dba6435cb WatchSource:0}: Error finding container fd10218e822b7d84dd08da69e3d78d002adf0e2c7e5e7491f4b7b38dba6435cb: Status 404 returned error can't find the container with id fd10218e822b7d84dd08da69e3d78d002adf0e2c7e5e7491f4b7b38dba6435cb
Oct 30 18:57:31.679112 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.679075    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6" event={"ID":"4c31cd17-e19d-43a0-9a77-f6819eae43fb","Type":"ContainerStarted","Data":"fd10218e822b7d84dd08da69e3d78d002adf0e2c7e5e7491f4b7b38dba6435cb"}
Oct 30 18:57:31.690369 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:31.690342    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4"
Oct 30 18:57:32.686081 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:32.686043    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6" event={"ID":"4c31cd17-e19d-43a0-9a77-f6819eae43fb","Type":"ContainerStarted","Data":"358c4a342a3532c37c2b55a665fa4fa17ded4624518358d4b346bfbbed95a3c4"}
Oct 30 18:57:32.802179 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:32.802126    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6" podStartSLOduration=2.119863225 podStartE2EDuration="2.802112163s" totalImagesPullingTime="682.248938ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:32.801109445 +0000 UTC m=+645.385592443" watchObservedRunningTime="2026-10-30 18:57:32.802112163 +0000 UTC m=+645.386595139"
Oct 30 18:57:33.202502 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:33.202468    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:37.937573 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:37.937534    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"]
Oct 30 18:57:37.941540 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:37.941516    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.039245 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.039216    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 18:57:38.041166 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.041141    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"]
Oct 30 18:57:38.073366 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.073323    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6d634df-343d-47bd-b07c-9d180268a5c7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.073366 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.073360    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsfjb\" (UniqueName: \"kubernetes.io/projected/e6d634df-343d-47bd-b07c-9d180268a5c7-kube-api-access-nsfjb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.073640 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.073391    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.073640 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.073524    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.073640 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.073606    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.073758 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.073680    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.174732 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.174689    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6d634df-343d-47bd-b07c-9d180268a5c7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.174984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.174741    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsfjb\" (UniqueName: \"kubernetes.io/projected/e6d634df-343d-47bd-b07c-9d180268a5c7-kube-api-access-nsfjb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.174984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.174772    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.174984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.174795    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.174984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.174843    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.174984 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.174891    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.175317 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.175288    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.175384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.175300    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.175384 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.175357    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.177079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.177051    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6d634df-343d-47bd-b07c-9d180268a5c7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.177452 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.177429    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6d634df-343d-47bd-b07c-9d180268a5c7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.234103 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.234025    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsfjb\" (UniqueName: \"kubernetes.io/projected/e6d634df-343d-47bd-b07c-9d180268a5c7-kube-api-access-nsfjb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx\" (UID: \"e6d634df-343d-47bd-b07c-9d180268a5c7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.253744 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.253707    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:38.632296 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.632263    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"]
Oct 30 18:57:38.637569 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:57:38.637541    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d634df_343d_47bd_b07c_9d180268a5c7.slice/crio-07fff6cddc49151ad0523312ae82d4bdc2373ecf28289dba8d6195248fa8b11e WatchSource:0}: Error finding container 07fff6cddc49151ad0523312ae82d4bdc2373ecf28289dba8d6195248fa8b11e: Status 404 returned error can't find the container with id 07fff6cddc49151ad0523312ae82d4bdc2373ecf28289dba8d6195248fa8b11e
Oct 30 18:57:38.718458 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:38.718421    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx" event={"ID":"e6d634df-343d-47bd-b07c-9d180268a5c7","Type":"ContainerStarted","Data":"07fff6cddc49151ad0523312ae82d4bdc2373ecf28289dba8d6195248fa8b11e"}
Oct 30 18:57:39.724352 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:39.724316    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx" event={"ID":"e6d634df-343d-47bd-b07c-9d180268a5c7","Type":"ContainerStarted","Data":"59a350756dd5ca62b05aa4ff871bb485c641b456d0ca0bdaeaa21438c938f5f5"}
Oct 30 18:57:39.840878 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:39.840827    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx" podStartSLOduration=2.248196165 podStartE2EDuration="2.84079632s" totalImagesPullingTime="592.600155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:39.839639295 +0000 UTC m=+652.424122291" watchObservedRunningTime="2026-10-30 18:57:39.84079632 +0000 UTC m=+652.425279317"
Oct 30 18:57:40.254221 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:40.254186    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:41.202545 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:41.202492    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:41.220561 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:41.220525    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:41.755456 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:41.755185    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wtpr6"
Oct 30 18:57:45.029349 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.029314    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"]
Oct 30 18:57:45.032974 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.032953    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.130748 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.130716    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 30 18:57:45.132874 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.132849    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"]
Oct 30 18:57:45.145709 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.145681    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.145801 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.145713    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.145801 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.145731    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/382fe2e8-957d-4996-a946-aeb825d0d31c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.145801 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.145747    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.145953 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.145931    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.145992 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.145967    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48hjv\" (UniqueName: \"kubernetes.io/projected/382fe2e8-957d-4996-a946-aeb825d0d31c-kube-api-access-48hjv\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.246730 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.246696    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.246931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.246735    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.246931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.246755    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/382fe2e8-957d-4996-a946-aeb825d0d31c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.246931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.246774    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.246931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.246848    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.246931 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.246878    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48hjv\" (UniqueName: \"kubernetes.io/projected/382fe2e8-957d-4996-a946-aeb825d0d31c-kube-api-access-48hjv\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.247156 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.247141    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.247218 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.247194    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.247332 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.247305    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.249183 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.249160    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/382fe2e8-957d-4996-a946-aeb825d0d31c-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.249380 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.249361    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/382fe2e8-957d-4996-a946-aeb825d0d31c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.305703 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.305634    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48hjv\" (UniqueName: \"kubernetes.io/projected/382fe2e8-957d-4996-a946-aeb825d0d31c-kube-api-access-48hjv\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr\" (UID: \"382fe2e8-957d-4996-a946-aeb825d0d31c\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.345572 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.345533    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:45.720256 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.720222    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"]
Oct 30 18:57:45.725514 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:57:45.725486    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod382fe2e8_957d_4996_a946_aeb825d0d31c.slice/crio-dff01b368c2de3833ca5f784255b21b71ccbda8cb0a804ecf7995c4d71fea6cf WatchSource:0}: Error finding container dff01b368c2de3833ca5f784255b21b71ccbda8cb0a804ecf7995c4d71fea6cf: Status 404 returned error can't find the container with id dff01b368c2de3833ca5f784255b21b71ccbda8cb0a804ecf7995c4d71fea6cf
Oct 30 18:57:45.757711 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:45.757674    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr" event={"ID":"382fe2e8-957d-4996-a946-aeb825d0d31c","Type":"ContainerStarted","Data":"dff01b368c2de3833ca5f784255b21b71ccbda8cb0a804ecf7995c4d71fea6cf"}
Oct 30 18:57:46.186090 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:57:46.186051    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:57:46.764114 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:46.764075    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr" event={"ID":"382fe2e8-957d-4996-a946-aeb825d0d31c","Type":"ContainerStarted","Data":"15d85ddf6ee01db6b96ff936a718c86672012462297c1e9daae1911262648c28"}
Oct 30 18:57:46.879374 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:46.879322    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr" podStartSLOduration=2.2570752020000002 podStartE2EDuration="2.879308589s" totalImagesPullingTime="622.233387ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:46.878091185 +0000 UTC m=+659.462574184" watchObservedRunningTime="2026-10-30 18:57:46.879308589 +0000 UTC m=+659.463791586"
Oct 30 18:57:47.345859 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:47.345807    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:48.254642 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:48.254599    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:48.267542 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:48.267509    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:48.792558 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:48.792528    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx"
Oct 30 18:57:55.346487 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:55.346445    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:55.359015 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:55.358983    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:57:55.822772 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:57:55.822702    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr"
Oct 30 18:58:11.389004 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.388974    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"]
Oct 30 18:58:11.392581 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.392559    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494145 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494113    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494145 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494147    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494380 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494177    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2716fd62-07ab-4d55-a7a4-41610ef78a12-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494380 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494289    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494380 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494327    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494380 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494347    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494533 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494425    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nthbc\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-kube-api-access-nthbc\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494533 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494450    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.494533 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.494467    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.593898 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.593864    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"]
Oct 30 18:58:11.595383 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595352    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.595522 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595428    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.595522 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595494    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2716fd62-07ab-4d55-a7a4-41610ef78a12-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.595638 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595558    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.595703 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595657    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.595703 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595694    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.595937 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595908    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nthbc\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-kube-api-access-nthbc\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.596038 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595957    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.596038 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.595986    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.596678 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.596653    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.597370 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.597342    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.597370 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.597336    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.597926 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.597728    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.597926 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.597890    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2716fd62-07ab-4d55-a7a4-41610ef78a12-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.598428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.598405    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.599222 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.599198    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.655533 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.655459    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.655680 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.655555    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nthbc\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-kube-api-access-nthbc\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-t42hj\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:11.705195 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:11.705166    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:12.177852 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:12.177804    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"]
Oct 30 18:58:12.182714 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:58:12.182683    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2716fd62_07ab_4d55_a7a4_41610ef78a12.slice/crio-1f88d976e0da9c21ffa9c5febd56dc1199930f537879bbeee18a27770f38f83b WatchSource:0}: Error finding container 1f88d976e0da9c21ffa9c5febd56dc1199930f537879bbeee18a27770f38f83b: Status 404 returned error can't find the container with id 1f88d976e0da9c21ffa9c5febd56dc1199930f537879bbeee18a27770f38f83b
Oct 30 18:58:12.184743 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:12.184709    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:58:12.184805 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:12.184780    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:58:12.184874 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:12.184829    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:58:12.903374 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:12.903339    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" event={"ID":"2716fd62-07ab-4d55-a7a4-41610ef78a12","Type":"ContainerStarted","Data":"1880ba3bafbb0c15ac3a249f4912afd07bb7436d8e3b44ce8452131fb88814dd"}
Oct 30 18:58:12.903780 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:12.903391    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" event={"ID":"2716fd62-07ab-4d55-a7a4-41610ef78a12","Type":"ContainerStarted","Data":"1f88d976e0da9c21ffa9c5febd56dc1199930f537879bbeee18a27770f38f83b"}
Oct 30 18:58:13.122162 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:13.122104    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" podStartSLOduration=2.122091077 podStartE2EDuration="2.122091077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:13.073136969 +0000 UTC m=+685.657619967" watchObservedRunningTime="2026-10-30 18:58:13.122091077 +0000 UTC m=+685.706574074"
Oct 30 18:58:13.706366 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:13.706330    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:13.707784 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:13.707762    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-t42hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 18:58:13.707861 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:13.707825    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 18:58:14.706392 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:14.706351    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-t42hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 18:58:14.706804 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:14.706416    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 18:58:15.705744 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:15.705705    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-t42hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 18:58:15.705946 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:15.705761    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 18:58:16.187378 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:58:16.187342    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:58:16.705676 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:16.705641    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-t42hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 18:58:16.705960 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:16.705695    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 18:58:17.706146 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:17.706109    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-t42hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 18:58:17.706552 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:17.706171    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 18:58:18.706370 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:18.706331    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-t42hj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 18:58:18.706872 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:18.706392    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 18:58:20.352133 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.352089    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:20.352796 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.352771    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:20.353454 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.353433    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 18:58:20.502721 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.502689    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58dcb89d9-lwzcs"]
Oct 30 18:58:20.506599 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.506579    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58dcb89d9-lwzcs"
Oct 30 18:58:20.565887 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.565850    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58dcb89d9-lwzcs"]
Oct 30 18:58:20.605298 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.605209    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q89w\" (UniqueName: \"kubernetes.io/projected/1a8f6a91-c365-409e-9eea-902eedd65d96-kube-api-access-5q89w\") pod \"authorino-58dcb89d9-lwzcs\" (UID: \"1a8f6a91-c365-409e-9eea-902eedd65d96\") " pod="kuadrant-system/authorino-58dcb89d9-lwzcs"
Oct 30 18:58:20.605298 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.605253    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a8f6a91-c365-409e-9eea-902eedd65d96-tls-cert\") pod \"authorino-58dcb89d9-lwzcs\" (UID: \"1a8f6a91-c365-409e-9eea-902eedd65d96\") " pod="kuadrant-system/authorino-58dcb89d9-lwzcs"
Oct 30 18:58:20.705952 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.705919    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q89w\" (UniqueName: \"kubernetes.io/projected/1a8f6a91-c365-409e-9eea-902eedd65d96-kube-api-access-5q89w\") pod \"authorino-58dcb89d9-lwzcs\" (UID: \"1a8f6a91-c365-409e-9eea-902eedd65d96\") " pod="kuadrant-system/authorino-58dcb89d9-lwzcs"
Oct 30 18:58:20.706155 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.705971    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a8f6a91-c365-409e-9eea-902eedd65d96-tls-cert\") pod \"authorino-58dcb89d9-lwzcs\" (UID: \"1a8f6a91-c365-409e-9eea-902eedd65d96\") " pod="kuadrant-system/authorino-58dcb89d9-lwzcs"
Oct 30 18:58:20.708589 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.708563    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a8f6a91-c365-409e-9eea-902eedd65d96-tls-cert\") pod \"authorino-58dcb89d9-lwzcs\" (UID: \"1a8f6a91-c365-409e-9eea-902eedd65d96\") " pod="kuadrant-system/authorino-58dcb89d9-lwzcs"
Oct 30 18:58:20.768342 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.768307    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q89w\" (UniqueName: \"kubernetes.io/projected/1a8f6a91-c365-409e-9eea-902eedd65d96-kube-api-access-5q89w\") pod \"authorino-58dcb89d9-lwzcs\" (UID: \"1a8f6a91-c365-409e-9eea-902eedd65d96\") " pod="kuadrant-system/authorino-58dcb89d9-lwzcs"
Oct 30 18:58:20.819222 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:20.819193    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58dcb89d9-lwzcs"
Oct 30 18:58:21.189958 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:21.189876    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58dcb89d9-lwzcs"]
Oct 30 18:58:21.195658 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:58:21.195628    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a8f6a91_c365_409e_9eea_902eedd65d96.slice/crio-b97c3c193b85bbc386963f8ccd4312b7d953dd17185215caab289982b004ba44 WatchSource:0}: Error finding container b97c3c193b85bbc386963f8ccd4312b7d953dd17185215caab289982b004ba44: Status 404 returned error can't find the container with id b97c3c193b85bbc386963f8ccd4312b7d953dd17185215caab289982b004ba44
Oct 30 18:58:21.950179 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:21.950140    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58dcb89d9-lwzcs" event={"ID":"1a8f6a91-c365-409e-9eea-902eedd65d96","Type":"ContainerStarted","Data":"b97c3c193b85bbc386963f8ccd4312b7d953dd17185215caab289982b004ba44"}
Oct 30 18:58:22.957243 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:22.957203    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58dcb89d9-lwzcs" event={"ID":"1a8f6a91-c365-409e-9eea-902eedd65d96","Type":"ContainerStarted","Data":"579a4ba5021a3a58aafe03fade495f976cb40934c2c877bf5b141000a7e730a8"}
Oct 30 18:58:23.068841 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.068706    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-58dcb89d9-lwzcs" podStartSLOduration=1.510477063 podStartE2EDuration="3.068692467s" totalImagesPullingTime="1.558215404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:23.067585834 +0000 UTC m=+695.652068828" watchObservedRunningTime="2026-10-30 18:58:23.068692467 +0000 UTC m=+695.653175463"
Oct 30 18:58:23.117542 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.117506    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8558845494-2hrn5"]
Oct 30 18:58:23.117713 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.117694    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-8558845494-2hrn5" podUID="04be873e-1f09-4680-be2a-5d7f3e7d5235" containerName="authorino" containerID="cri-o://493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58" gracePeriod=30
Oct 30 18:58:23.387282 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.387253    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:58:23.433958 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.433910    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04be873e-1f09-4680-be2a-5d7f3e7d5235-kube-api-access-6qx6c\" (UniqueName: \"kubernetes.io/projected/04be873e-1f09-4680-be2a-5d7f3e7d5235-kube-api-access-6qx6c\") pod \"04be873e-1f09-4680-be2a-5d7f3e7d5235\" (UID: \"04be873e-1f09-4680-be2a-5d7f3e7d5235\") "
Oct 30 18:58:23.433958 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.433959    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/04be873e-1f09-4680-be2a-5d7f3e7d5235-tls-cert\" (UniqueName: \"kubernetes.io/secret/04be873e-1f09-4680-be2a-5d7f3e7d5235-tls-cert\") pod \"04be873e-1f09-4680-be2a-5d7f3e7d5235\" (UID: \"04be873e-1f09-4680-be2a-5d7f3e7d5235\") "
Oct 30 18:58:23.436080 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.436046    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04be873e-1f09-4680-be2a-5d7f3e7d5235-kube-api-access-6qx6c" pod "04be873e-1f09-4680-be2a-5d7f3e7d5235" (UID: "04be873e-1f09-4680-be2a-5d7f3e7d5235"). InnerVolumeSpecName "kube-api-access-6qx6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:23.443966 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.443939    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04be873e-1f09-4680-be2a-5d7f3e7d5235-tls-cert" pod "04be873e-1f09-4680-be2a-5d7f3e7d5235" (UID: "04be873e-1f09-4680-be2a-5d7f3e7d5235"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:58:23.534919 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.534876    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qx6c\" (UniqueName: \"kubernetes.io/projected/04be873e-1f09-4680-be2a-5d7f3e7d5235-kube-api-access-6qx6c\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:23.535116 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.534922    5232 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/04be873e-1f09-4680-be2a-5d7f3e7d5235-tls-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:23.963539 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.963501    5232 generic.go:356] "Generic (PLEG): container finished" podID="04be873e-1f09-4680-be2a-5d7f3e7d5235" containerID="493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58" exitCode=0
Oct 30 18:58:23.964060 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.963544    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8558845494-2hrn5"
Oct 30 18:58:23.964060 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.963593    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8558845494-2hrn5" event={"ID":"04be873e-1f09-4680-be2a-5d7f3e7d5235","Type":"ContainerDied","Data":"493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58"}
Oct 30 18:58:23.964060 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.963637    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8558845494-2hrn5" event={"ID":"04be873e-1f09-4680-be2a-5d7f3e7d5235","Type":"ContainerDied","Data":"ce413e07fa8bce53545f96a28983e22eb07a34da24900a7edd321bf51f45b40b"}
Oct 30 18:58:23.964060 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.963657    5232 scope.go:122] "RemoveContainer" containerID="493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58"
Oct 30 18:58:23.974429 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.974409    5232 scope.go:122] "RemoveContainer" containerID="493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58"
Oct 30 18:58:23.974715 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:58:23.974693    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58\": container with ID starting with 493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58 not found: ID does not exist" containerID="493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58"
Oct 30 18:58:23.974801 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:23.974727    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58"} err="failed to get container status \"493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58\": rpc error: code = NotFound desc = could not find container \"493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58\": container with ID starting with 493c0cc32d54515302b39d29e04c7d4a5e3bd97187e2876a533ac5c30bd0cc58 not found: ID does not exist"
Oct 30 18:58:24.120291 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:24.120255    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8558845494-2hrn5"]
Oct 30 18:58:24.168010 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:24.167977    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8558845494-2hrn5"]
Oct 30 18:58:25.538182 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:58:25.538151    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04be873e-1f09-4680-be2a-5d7f3e7d5235" path="/var/lib/kubelet/pods/04be873e-1f09-4680-be2a-5d7f3e7d5235/volumes"
Oct 30 18:58:46.187403 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:58:46.187362    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:59:16.186532 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:59:16.186486    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:59:43.814331 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.814292    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"]
Oct 30 18:59:43.815062 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.814914    5232 cpu_manager.go:414] "containerMap: removing container" podUID="04be873e-1f09-4680-be2a-5d7f3e7d5235" containerName="authorino"
Oct 30 18:59:43.815062 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.814933    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04be873e-1f09-4680-be2a-5d7f3e7d5235" containerName="authorino"
Oct 30 18:59:43.815062 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.815028    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="04be873e-1f09-4680-be2a-5d7f3e7d5235" containerName="authorino"
Oct 30 18:59:43.818930 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.818907    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.880326 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.880287    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e7b96f88-openshift-default-dockercfg-7jgcd\""
Oct 30 18:59:43.903004 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.902971    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-credential-socket\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.903178 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.903023    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-token\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.903178 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.903129    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-data\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.903178 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.903170    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-socket\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.903330 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.903191    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7xjs\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-kube-api-access-x7xjs\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.903330 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.903310    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-podinfo\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.903440 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.903340    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-envoy\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.903440 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.903359    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-certs\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.903440 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.903374    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c5e8235-3b0b-4684-b9c8-7438399620ab-istiod-ca-cert\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:43.939154 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:43.939121    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"]
Oct 30 18:59:44.004715 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.004673    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-data\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.004912 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.004728    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-socket\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.004912 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.004862    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7xjs\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-kube-api-access-x7xjs\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.004994 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.004939    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-podinfo\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.004994 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.004965    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-envoy\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.004994 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.004984    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-certs\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.005139 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.005007    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c5e8235-3b0b-4684-b9c8-7438399620ab-istiod-ca-cert\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.005139 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.005059    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-credential-socket\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.005139 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.005102    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-token\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.005139 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.005107    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-socket\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.005342 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.005149    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-data\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.005430 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.005401    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-certs\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.005430 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.005422    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-credential-socket\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.005770 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.005748    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c5e8235-3b0b-4684-b9c8-7438399620ab-istiod-ca-cert\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.007218 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.007194    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-envoy\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.007415 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.007397    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-podinfo\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.066514 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.066443    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-token\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.067006 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.066988    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7xjs\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-kube-api-access-x7xjs\") pod \"e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") " pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.135873 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.135799    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:44.496885 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.496847    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"]
Oct 30 18:59:44.500722 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.500701    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.551377 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.551348    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-45e6849e-gw-openshift-default-dockercfg-v47hx\""
Oct 30 18:59:44.562428 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.562403    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"]
Oct 30 18:59:44.611203 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611171    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-data\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.611371 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611209    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-certs\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.611371 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611236    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-podinfo\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.611371 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611338    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvw2v\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-kube-api-access-lvw2v\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.611487 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611401    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-token\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.611487 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611436    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-envoy\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.611487 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611463    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istiod-ca-cert\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.611589 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611532    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-credential-socket\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.611589 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.611552    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-socket\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.617718 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.617690    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"]
Oct 30 18:59:44.623326 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:59:44.623297    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5e8235_3b0b_4684_b9c8_7438399620ab.slice/crio-54f79cd27fa11b5375bff0adbbde47cb68fe390598bbb29e363d094bd425ba44 WatchSource:0}: Error finding container 54f79cd27fa11b5375bff0adbbde47cb68fe390598bbb29e363d094bd425ba44: Status 404 returned error can't find the container with id 54f79cd27fa11b5375bff0adbbde47cb68fe390598bbb29e363d094bd425ba44
Oct 30 18:59:44.625490 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.625463    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:44.625547 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.625516    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:44.625589 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.625546    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:44.713089 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713051    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-certs\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713266 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713110    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-podinfo\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713266 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713250    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvw2v\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-kube-api-access-lvw2v\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713408 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713284    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-token\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713408 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713330    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-envoy\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713532 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713508    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-certs\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713588 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713569    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istiod-ca-cert\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713652 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713638    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-credential-socket\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713724 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713668    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-socket\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.713780 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.713752    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-data\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.714052 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.714021    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-credential-socket\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.714113 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.714072    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-data\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.714267 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.714247    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-socket\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.714682 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.714662    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istiod-ca-cert\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.716336 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.716314    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-podinfo\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.716570 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.716553    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-envoy\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.774005 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.773887    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-token\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.774866 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.774837    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvw2v\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-kube-api-access-lvw2v\") pod \"e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") " pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:44.814523 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:44.814493    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:45.242004 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.241968    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"]
Oct 30 18:59:45.247401 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:59:45.247373    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09fbe39_3d63_4b7e_843f_83bfabd433fb.slice/crio-9c95f5fad671bd3a84df0f17e491ddaab019b798614ee19775a2ea3e68c9fed5 WatchSource:0}: Error finding container 9c95f5fad671bd3a84df0f17e491ddaab019b798614ee19775a2ea3e68c9fed5: Status 404 returned error can't find the container with id 9c95f5fad671bd3a84df0f17e491ddaab019b798614ee19775a2ea3e68c9fed5
Oct 30 18:59:45.249654 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.249620    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:45.249743 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.249688    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:45.249743 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.249716    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:45.368079 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.368028    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" event={"ID":"a09fbe39-3d63-4b7e-843f-83bfabd433fb","Type":"ContainerStarted","Data":"ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675"}
Oct 30 18:59:45.368247 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.368102    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" event={"ID":"a09fbe39-3d63-4b7e-843f-83bfabd433fb","Type":"ContainerStarted","Data":"9c95f5fad671bd3a84df0f17e491ddaab019b798614ee19775a2ea3e68c9fed5"}
Oct 30 18:59:45.369828 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.369787    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" event={"ID":"7c5e8235-3b0b-4684-b9c8-7438399620ab","Type":"ContainerStarted","Data":"eaef2c5d17fbcc30ca48ef21bbdf448e012ffc7952b2512ea929725f2e2ea3e7"}
Oct 30 18:59:45.369943 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.369855    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" event={"ID":"7c5e8235-3b0b-4684-b9c8-7438399620ab","Type":"ContainerStarted","Data":"54f79cd27fa11b5375bff0adbbde47cb68fe390598bbb29e363d094bd425ba44"}
Oct 30 18:59:45.487650 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.487580    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" podStartSLOduration=1.487562095 podStartE2EDuration="1.487562095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:45.48533112 +0000 UTC m=+778.069814118" watchObservedRunningTime="2026-10-30 18:59:45.487562095 +0000 UTC m=+778.072045094"
Oct 30 18:59:45.605469 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.605407    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" podStartSLOduration=2.605384582 podStartE2EDuration="2.605384582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:45.602028652 +0000 UTC m=+778.186511650" watchObservedRunningTime="2026-10-30 18:59:45.605384582 +0000 UTC m=+778.189867581"
Oct 30 18:59:45.815518 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:45.815490    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:46.136135 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:46.136101    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:46.141219 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:46.141189    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:46.188210 ip-10-0-143-248 kubenswrapper[5232]: E1030 18:59:46.188173    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 18:59:46.375599 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:46.375566    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:46.376418 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:46.376395    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 18:59:46.820645 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:46.820618    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:47.380731 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:47.380696    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:47.381449 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:47.381431    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 18:59:51.835412 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.835378    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"]
Oct 30 18:59:51.840348 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.840325    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:51.881262 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.881223    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl2vc\" (UniqueName: \"kubernetes.io/projected/3eab5908-871d-4de6-a981-8ac079ebe40b-kube-api-access-hl2vc\") pod \"payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds\" (UID: \"3eab5908-871d-4de6-a981-8ac079ebe40b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:51.881430 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.881281    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eab5908-871d-4de6-a981-8ac079ebe40b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds\" (UID: \"3eab5908-871d-4de6-a981-8ac079ebe40b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:51.890550 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.890522    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-45e6849e\""
Oct 30 18:59:51.943497 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.943463    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"]
Oct 30 18:59:51.943649 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.943527    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-45e6849e-dockercfg-5nrxn\""
Oct 30 18:59:51.982974 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.982930    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl2vc\" (UniqueName: \"kubernetes.io/projected/3eab5908-871d-4de6-a981-8ac079ebe40b-kube-api-access-hl2vc\") pod \"payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds\" (UID: \"3eab5908-871d-4de6-a981-8ac079ebe40b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:51.983180 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.983000    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eab5908-871d-4de6-a981-8ac079ebe40b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds\" (UID: \"3eab5908-871d-4de6-a981-8ac079ebe40b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:51.983800 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:51.983774    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eab5908-871d-4de6-a981-8ac079ebe40b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds\" (UID: \"3eab5908-871d-4de6-a981-8ac079ebe40b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:52.041949 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:52.041914    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl2vc\" (UniqueName: \"kubernetes.io/projected/3eab5908-871d-4de6-a981-8ac079ebe40b-kube-api-access-hl2vc\") pod \"payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds\" (UID: \"3eab5908-871d-4de6-a981-8ac079ebe40b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:52.153180 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:52.153140    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:52.525286 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:52.525184    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"]
Oct 30 18:59:52.530053 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:59:52.530020    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eab5908_871d_4de6_a981_8ac079ebe40b.slice/crio-e25ccf21a75e3a4bfd028d4425141412e0a18340dccc4c0d0f379561de26d6d8 WatchSource:0}: Error finding container e25ccf21a75e3a4bfd028d4425141412e0a18340dccc4c0d0f379561de26d6d8: Status 404 returned error can't find the container with id e25ccf21a75e3a4bfd028d4425141412e0a18340dccc4c0d0f379561de26d6d8
Oct 30 18:59:53.415059 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.415024    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds" event={"ID":"3eab5908-871d-4de6-a981-8ac079ebe40b","Type":"ContainerStarted","Data":"25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666"}
Oct 30 18:59:53.415453 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.415081    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds" event={"ID":"3eab5908-871d-4de6-a981-8ac079ebe40b","Type":"ContainerStarted","Data":"e25ccf21a75e3a4bfd028d4425141412e0a18340dccc4c0d0f379561de26d6d8"}
Oct 30 18:59:53.415453 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.415162    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 18:59:53.528848 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.528778    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds" podStartSLOduration=2.528763754 podStartE2EDuration="2.528763754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:53.527012211 +0000 UTC m=+786.111495208" watchObservedRunningTime="2026-10-30 18:59:53.528763754 +0000 UTC m=+786.113246751"
Oct 30 18:59:53.709095 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.709019    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"]
Oct 30 18:59:53.712696 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.712667    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.763750 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.763710    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e7b96f88-serving-cert\""
Oct 30 18:59:53.763935 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.763866    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7w45v\""
Oct 30 18:59:53.763935 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.763885    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 18:59:53.772889 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.772863    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"]
Oct 30 18:59:53.802379 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.802344    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-metrics-certs\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.802544 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.802391    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-maas-api-tls\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.802544 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.802477    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkcvp\" (UniqueName: \"kubernetes.io/projected/93321856-90e6-4fc5-96a5-5d879a432501-kube-api-access-pkcvp\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.802666 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.802642    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93321856-90e6-4fc5-96a5-5d879a432501-openshift-service-ca\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.904020 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.903979    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkcvp\" (UniqueName: \"kubernetes.io/projected/93321856-90e6-4fc5-96a5-5d879a432501-kube-api-access-pkcvp\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.904174 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.904078    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93321856-90e6-4fc5-96a5-5d879a432501-openshift-service-ca\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.904174 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.904109    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-metrics-certs\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.904174 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.904138    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-maas-api-tls\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.904867 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.904838    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93321856-90e6-4fc5-96a5-5d879a432501-openshift-service-ca\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.906661 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.906635    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-maas-api-tls\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.906739 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.906666    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-metrics-certs\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:53.964216 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:53.964145    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkcvp\" (UniqueName: \"kubernetes.io/projected/93321856-90e6-4fc5-96a5-5d879a432501-kube-api-access-pkcvp\") pod \"maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:54.025775 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:54.025734    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:54.402581 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:54.402542    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"]
Oct 30 18:59:54.409258 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:59:54.409229    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93321856_90e6_4fc5_96a5_5d879a432501.slice/crio-266ff64cdd44c1dc9bc325a5db8413af5b9fe113e5d385e80730bb9c8fec4d94 WatchSource:0}: Error finding container 266ff64cdd44c1dc9bc325a5db8413af5b9fe113e5d385e80730bb9c8fec4d94: Status 404 returned error can't find the container with id 266ff64cdd44c1dc9bc325a5db8413af5b9fe113e5d385e80730bb9c8fec4d94
Oct 30 18:59:54.420941 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:54.420912    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh" event={"ID":"93321856-90e6-4fc5-96a5-5d879a432501","Type":"ContainerStarted","Data":"266ff64cdd44c1dc9bc325a5db8413af5b9fe113e5d385e80730bb9c8fec4d94"}
Oct 30 18:59:55.428980 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:55.428943    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh" event={"ID":"93321856-90e6-4fc5-96a5-5d879a432501","Type":"ContainerStarted","Data":"8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586"}
Oct 30 18:59:55.429378 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:55.429065    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 18:59:55.588353 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:55.588305    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh" podStartSLOduration=1.6872084859999998 podStartE2EDuration="2.588290262s" totalImagesPullingTime="901.081776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:55.585724018 +0000 UTC m=+788.170207017" watchObservedRunningTime="2026-10-30 18:59:55.588290262 +0000 UTC m=+788.172773260"
Oct 30 18:59:56.439118 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.439080    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"]
Oct 30 18:59:56.443198 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.443175    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:56.509011 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.508975    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b4e8ca3e-dockercfg-xbfqc\""
Oct 30 18:59:56.509673 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.509651    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b4e8ca3e\""
Oct 30 18:59:56.522883 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.520493    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"]
Oct 30 18:59:56.535906 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.535865    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4a5b01f-5be8-49aa-834f-c762a0ac8583-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq\" (UID: \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:56.536082 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.535910    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbh7b\" (UniqueName: \"kubernetes.io/projected/b4a5b01f-5be8-49aa-834f-c762a0ac8583-kube-api-access-wbh7b\") pod \"payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq\" (UID: \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:56.607462 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.607431    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"]
Oct 30 18:59:56.611306 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.611284    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:56.637056 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.637026    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4a5b01f-5be8-49aa-834f-c762a0ac8583-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq\" (UID: \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:56.637056 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.637057    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbh7b\" (UniqueName: \"kubernetes.io/projected/b4a5b01f-5be8-49aa-834f-c762a0ac8583-kube-api-access-wbh7b\") pod \"payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq\" (UID: \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:56.637769 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.637749    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4a5b01f-5be8-49aa-834f-c762a0ac8583-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq\" (UID: \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:56.672105 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.672011    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"]
Oct 30 18:59:56.696844 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.696750    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbh7b\" (UniqueName: \"kubernetes.io/projected/b4a5b01f-5be8-49aa-834f-c762a0ac8583-kube-api-access-wbh7b\") pod \"payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq\" (UID: \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:56.737785 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.737745    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5896ae3-0199-4a33-8818-ef2317ab1c43-plugins-config-volume\") pod \"payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5\" (UID: \"b5896ae3-0199-4a33-8818-ef2317ab1c43\") " pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:56.737978 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.737897    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfmg7\" (UniqueName: \"kubernetes.io/projected/b5896ae3-0199-4a33-8818-ef2317ab1c43-kube-api-access-kfmg7\") pod \"payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5\" (UID: \"b5896ae3-0199-4a33-8818-ef2317ab1c43\") " pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:56.756688 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.756653    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:56.852404 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.841929    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5896ae3-0199-4a33-8818-ef2317ab1c43-plugins-config-volume\") pod \"payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5\" (UID: \"b5896ae3-0199-4a33-8818-ef2317ab1c43\") " pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:56.852404 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.842045    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfmg7\" (UniqueName: \"kubernetes.io/projected/b5896ae3-0199-4a33-8818-ef2317ab1c43-kube-api-access-kfmg7\") pod \"payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5\" (UID: \"b5896ae3-0199-4a33-8818-ef2317ab1c43\") " pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:56.852404 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.843392    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5896ae3-0199-4a33-8818-ef2317ab1c43-plugins-config-volume\") pod \"payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5\" (UID: \"b5896ae3-0199-4a33-8818-ef2317ab1c43\") " pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:56.906101 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.906067    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfmg7\" (UniqueName: \"kubernetes.io/projected/b5896ae3-0199-4a33-8818-ef2317ab1c43-kube-api-access-kfmg7\") pod \"payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5\" (UID: \"b5896ae3-0199-4a33-8818-ef2317ab1c43\") " pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:56.924678 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:56.924644    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:57.135744 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.135707    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"]
Oct 30 18:59:57.140168 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:59:57.140138    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a5b01f_5be8_49aa_834f_c762a0ac8583.slice/crio-9c385700f44c4b7569deeecfdbe7ff9f9112e1a67a164172452725a5b0c816e6 WatchSource:0}: Error finding container 9c385700f44c4b7569deeecfdbe7ff9f9112e1a67a164172452725a5b0c816e6: Status 404 returned error can't find the container with id 9c385700f44c4b7569deeecfdbe7ff9f9112e1a67a164172452725a5b0c816e6
Oct 30 18:59:57.302646 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.302566    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"]
Oct 30 18:59:57.308176 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:59:57.308144    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5896ae3_0199_4a33_8818_ef2317ab1c43.slice/crio-97f2d14a46ca299f1cb1c72e4369643dc2db35bcbd26ff99d6cf45468856fc3b WatchSource:0}: Error finding container 97f2d14a46ca299f1cb1c72e4369643dc2db35bcbd26ff99d6cf45468856fc3b: Status 404 returned error can't find the container with id 97f2d14a46ca299f1cb1c72e4369643dc2db35bcbd26ff99d6cf45468856fc3b
Oct 30 18:59:57.441777 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.441740    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5" event={"ID":"b5896ae3-0199-4a33-8818-ef2317ab1c43","Type":"ContainerStarted","Data":"11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49"}
Oct 30 18:59:57.442188 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.441799    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5" event={"ID":"b5896ae3-0199-4a33-8818-ef2317ab1c43","Type":"ContainerStarted","Data":"97f2d14a46ca299f1cb1c72e4369643dc2db35bcbd26ff99d6cf45468856fc3b"}
Oct 30 18:59:57.442188 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.441856    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 18:59:57.443387 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.443362    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq" event={"ID":"b4a5b01f-5be8-49aa-834f-c762a0ac8583","Type":"ContainerStarted","Data":"3fd9a0fbd703b021ef183ab6e312ec51f3d501c15e4343ce1742307197bddd22"}
Oct 30 18:59:57.443485 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.443406    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq" event={"ID":"b4a5b01f-5be8-49aa-834f-c762a0ac8583","Type":"ContainerStarted","Data":"9c385700f44c4b7569deeecfdbe7ff9f9112e1a67a164172452725a5b0c816e6"}
Oct 30 18:59:57.443485 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.443431    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 18:59:57.556111 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.556016    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5" podStartSLOduration=1.556003804 podStartE2EDuration="1.556003804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:57.554040868 +0000 UTC m=+790.138523868" watchObservedRunningTime="2026-10-30 18:59:57.556003804 +0000 UTC m=+790.140486799"
Oct 30 18:59:57.688145 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:57.688087    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq" podStartSLOduration=1.688074328 podStartE2EDuration="1.688074328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:57.686563934 +0000 UTC m=+790.271046933" watchObservedRunningTime="2026-10-30 18:59:57.688074328 +0000 UTC m=+790.272557325"
Oct 30 18:59:58.870948 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:58.870910    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"]
Oct 30 18:59:58.874708 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:58.874687    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 18:59:58.925473 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:58.925423    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e7b96f88\""
Oct 30 18:59:58.963696 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:58.963647    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b47854-adb1-4f64-ab2d-7aab1267570b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c\" (UID: \"e2b47854-adb1-4f64-ab2d-7aab1267570b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 18:59:58.964001 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:58.963848    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmkp4\" (UniqueName: \"kubernetes.io/projected/e2b47854-adb1-4f64-ab2d-7aab1267570b-kube-api-access-pmkp4\") pod \"payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c\" (UID: \"e2b47854-adb1-4f64-ab2d-7aab1267570b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 18:59:58.972807 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:58.972781    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e7b96f88-dockercfg-mzvt4\""
Oct 30 18:59:58.975286 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:58.975265    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"]
Oct 30 18:59:59.028848 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.028798    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"]
Oct 30 18:59:59.033725 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.033699    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 18:59:59.065138 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.065095    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b47854-adb1-4f64-ab2d-7aab1267570b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c\" (UID: \"e2b47854-adb1-4f64-ab2d-7aab1267570b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 18:59:59.065308 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.065189    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmkp4\" (UniqueName: \"kubernetes.io/projected/e2b47854-adb1-4f64-ab2d-7aab1267570b-kube-api-access-pmkp4\") pod \"payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c\" (UID: \"e2b47854-adb1-4f64-ab2d-7aab1267570b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 18:59:59.066050 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.066022    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b47854-adb1-4f64-ab2d-7aab1267570b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c\" (UID: \"e2b47854-adb1-4f64-ab2d-7aab1267570b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 18:59:59.090856 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.090824    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"]
Oct 30 18:59:59.124273 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.124188    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmkp4\" (UniqueName: \"kubernetes.io/projected/e2b47854-adb1-4f64-ab2d-7aab1267570b-kube-api-access-pmkp4\") pod \"payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c\" (UID: \"e2b47854-adb1-4f64-ab2d-7aab1267570b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 18:59:59.166587 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.166551    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65caf4dd-f404-4155-af77-5a1b8462fe2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj\" (UID: \"65caf4dd-f404-4155-af77-5a1b8462fe2d\") " pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 18:59:59.166776 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.166687    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhkvn\" (UniqueName: \"kubernetes.io/projected/65caf4dd-f404-4155-af77-5a1b8462fe2d-kube-api-access-dhkvn\") pod \"payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj\" (UID: \"65caf4dd-f404-4155-af77-5a1b8462fe2d\") " pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 18:59:59.188450 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.188412    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 18:59:59.270080 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.268368    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65caf4dd-f404-4155-af77-5a1b8462fe2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj\" (UID: \"65caf4dd-f404-4155-af77-5a1b8462fe2d\") " pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 18:59:59.270080 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.268501    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhkvn\" (UniqueName: \"kubernetes.io/projected/65caf4dd-f404-4155-af77-5a1b8462fe2d-kube-api-access-dhkvn\") pod \"payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj\" (UID: \"65caf4dd-f404-4155-af77-5a1b8462fe2d\") " pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 18:59:59.270080 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.270002    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65caf4dd-f404-4155-af77-5a1b8462fe2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj\" (UID: \"65caf4dd-f404-4155-af77-5a1b8462fe2d\") " pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 18:59:59.328979 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.328947    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhkvn\" (UniqueName: \"kubernetes.io/projected/65caf4dd-f404-4155-af77-5a1b8462fe2d-kube-api-access-dhkvn\") pod \"payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj\" (UID: \"65caf4dd-f404-4155-af77-5a1b8462fe2d\") " pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 18:59:59.346789 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.346757    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 18:59:59.568865 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.568808    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"]
Oct 30 18:59:59.575039 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:59:59.575008    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b47854_adb1_4f64_ab2d_7aab1267570b.slice/crio-15610cccbfe7592d1a548f658702d7c93c011795b404e8c7561ad5de22c9f5b4 WatchSource:0}: Error finding container 15610cccbfe7592d1a548f658702d7c93c011795b404e8c7561ad5de22c9f5b4: Status 404 returned error can't find the container with id 15610cccbfe7592d1a548f658702d7c93c011795b404e8c7561ad5de22c9f5b4
Oct 30 18:59:59.724078 ip-10-0-143-248 kubenswrapper[5232]: I1030 18:59:59.723995    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"]
Oct 30 18:59:59.729263 ip-10-0-143-248 kubenswrapper[5232]: W1030 18:59:59.729229    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65caf4dd_f404_4155_af77_5a1b8462fe2d.slice/crio-4f73ce15cd781c851f2a0d0751e04578f7442454b548d50c9f2fd703f1356eef WatchSource:0}: Error finding container 4f73ce15cd781c851f2a0d0751e04578f7442454b548d50c9f2fd703f1356eef: Status 404 returned error can't find the container with id 4f73ce15cd781c851f2a0d0751e04578f7442454b548d50c9f2fd703f1356eef
Oct 30 19:00:00.177162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.177128    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"]
Oct 30 19:00:00.180905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.180885    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:00.238796 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.238763    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"]
Oct 30 19:00:00.279150 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.279118    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvgdb\" (UniqueName: \"kubernetes.io/projected/d1bda426-2b67-4462-9e10-ae982167c9f3-kube-api-access-lvgdb\") pod \"maas-api-key-cleanup-29889780-xr6qv\" (UID: \"d1bda426-2b67-4462-9e10-ae982167c9f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:00.279294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.279254    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bda426-2b67-4462-9e10-ae982167c9f3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889780-xr6qv\" (UID: \"d1bda426-2b67-4462-9e10-ae982167c9f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:00.325648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.325611    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 19:00:00.380426 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.380384    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bda426-2b67-4462-9e10-ae982167c9f3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889780-xr6qv\" (UID: \"d1bda426-2b67-4462-9e10-ae982167c9f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:00.380618 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.380479    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvgdb\" (UniqueName: \"kubernetes.io/projected/d1bda426-2b67-4462-9e10-ae982167c9f3-kube-api-access-lvgdb\") pod \"maas-api-key-cleanup-29889780-xr6qv\" (UID: \"d1bda426-2b67-4462-9e10-ae982167c9f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:00.381106 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.381084    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bda426-2b67-4462-9e10-ae982167c9f3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889780-xr6qv\" (UID: \"d1bda426-2b67-4462-9e10-ae982167c9f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:00.439627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.439548    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvgdb\" (UniqueName: \"kubernetes.io/projected/d1bda426-2b67-4462-9e10-ae982167c9f3-kube-api-access-lvgdb\") pod \"maas-api-key-cleanup-29889780-xr6qv\" (UID: \"d1bda426-2b67-4462-9e10-ae982167c9f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:00.471712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.471669    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj" event={"ID":"65caf4dd-f404-4155-af77-5a1b8462fe2d","Type":"ContainerStarted","Data":"e65c5eb6f71786ca7b8637cc389b1aeb9b3dccb9e06c0c6abf418cb004a82f3f"}
Oct 30 19:00:00.471948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.471773    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj" event={"ID":"65caf4dd-f404-4155-af77-5a1b8462fe2d","Type":"ContainerStarted","Data":"4f73ce15cd781c851f2a0d0751e04578f7442454b548d50c9f2fd703f1356eef"}
Oct 30 19:00:00.471948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.471801    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 19:00:00.473580 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.473544    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c" event={"ID":"e2b47854-adb1-4f64-ab2d-7aab1267570b","Type":"ContainerStarted","Data":"24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a"}
Oct 30 19:00:00.473712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.473605    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c" event={"ID":"e2b47854-adb1-4f64-ab2d-7aab1267570b","Type":"ContainerStarted","Data":"15610cccbfe7592d1a548f658702d7c93c011795b404e8c7561ad5de22c9f5b4"}
Oct 30 19:00:00.473712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.473657    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 19:00:00.494213 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.494171    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:00.587095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.587041    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj" podStartSLOduration=2.58702477 podStartE2EDuration="2.58702477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:00.5847359 +0000 UTC m=+793.169218896" watchObservedRunningTime="2026-10-30 19:00:00.58702477 +0000 UTC m=+793.171507768"
Oct 30 19:00:00.699931 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.699797    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c" podStartSLOduration=2.699782971 podStartE2EDuration="2.699782971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:00.699059706 +0000 UTC m=+793.283542704" watchObservedRunningTime="2026-10-30 19:00:00.699782971 +0000 UTC m=+793.284265967"
Oct 30 19:00:00.864852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:00.864803    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"]
Oct 30 19:00:00.870318 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:00:00.870289    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1bda426_2b67_4462_9e10_ae982167c9f3.slice/crio-09301615b5618ea05e8aa183372c2a8ae4c31c7f709dafbe427f2836a5c63a8a WatchSource:0}: Error finding container 09301615b5618ea05e8aa183372c2a8ae4c31c7f709dafbe427f2836a5c63a8a: Status 404 returned error can't find the container with id 09301615b5618ea05e8aa183372c2a8ae4c31c7f709dafbe427f2836a5c63a8a
Oct 30 19:00:01.440328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:01.440296    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 19:00:01.480597 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:01.480560    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" event={"ID":"d1bda426-2b67-4462-9e10-ae982167c9f3","Type":"ContainerStarted","Data":"09301615b5618ea05e8aa183372c2a8ae4c31c7f709dafbe427f2836a5c63a8a"}
Oct 30 19:00:04.014601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.014557    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"]
Oct 30 19:00:04.019907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.019877    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.118143 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.118109    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b4e8ca3e-serving-cert\""
Oct 30 19:00:04.120491 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.120459    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"]
Oct 30 19:00:04.128661 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.128631    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-metrics-certs\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.128851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.128671    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.128851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.128694    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a17886-d3bb-4b31-a671-bea255803eaa-openshift-service-ca\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.128851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.128771    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc6cf\" (UniqueName: \"kubernetes.io/projected/b4a17886-d3bb-4b31-a671-bea255803eaa-kube-api-access-rc6cf\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.229333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.229293    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-metrics-certs\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.229333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.229335    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.229603 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.229359    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a17886-d3bb-4b31-a671-bea255803eaa-openshift-service-ca\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.229603 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.229377    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc6cf\" (UniqueName: \"kubernetes.io/projected/b4a17886-d3bb-4b31-a671-bea255803eaa-kube-api-access-rc6cf\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.229603 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:04.229508    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-serving-cert: secret "maas-api-e2e-ait-b4e8ca3e-serving-cert" not found
Oct 30 19:00:04.229884 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:04.229605    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls podName:b4a17886-d3bb-4b31-a671-bea255803eaa nodeName:}" failed. No retries permitted until 2026-10-30 19:00:04.729580386 +0000 UTC m=+797.314063378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls") pod "maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc" (UID: "b4a17886-d3bb-4b31-a671-bea255803eaa") : secret "maas-api-e2e-ait-b4e8ca3e-serving-cert" not found
Oct 30 19:00:04.230110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.230081    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a17886-d3bb-4b31-a671-bea255803eaa-openshift-service-ca\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.231940 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.231922    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-metrics-certs\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.290102 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.290022    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc6cf\" (UniqueName: \"kubernetes.io/projected/b4a17886-d3bb-4b31-a671-bea255803eaa-kube-api-access-rc6cf\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.424222 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.424193    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 19:00:04.735029 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.734990    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.751454 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.751414    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls\") pod \"maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:04.935319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:04.935280    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:05.367205 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:05.364746    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"]
Oct 30 19:00:05.369906 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:00:05.369858    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a17886_d3bb_4b31_a671_bea255803eaa.slice/crio-c86b6986a40a5f013e7056968594f9d1f0685c8bf2ce7792cff7c5b404119a02 WatchSource:0}: Error finding container c86b6986a40a5f013e7056968594f9d1f0685c8bf2ce7792cff7c5b404119a02: Status 404 returned error can't find the container with id c86b6986a40a5f013e7056968594f9d1f0685c8bf2ce7792cff7c5b404119a02
Oct 30 19:00:05.572550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:05.572506    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc" event={"ID":"b4a17886-d3bb-4b31-a671-bea255803eaa","Type":"ContainerStarted","Data":"c86b6986a40a5f013e7056968594f9d1f0685c8bf2ce7792cff7c5b404119a02"}
Oct 30 19:00:06.570367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:06.570265    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc" event={"ID":"b4a17886-d3bb-4b31-a671-bea255803eaa","Type":"ContainerStarted","Data":"0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4"}
Oct 30 19:00:06.570850 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:06.570375    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:06.690043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:06.689985    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc" podStartSLOduration=2.746177878 podStartE2EDuration="3.689969078s" totalImagesPullingTime="943.7912ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:06.687049073 +0000 UTC m=+799.271532070" watchObservedRunningTime="2026-10-30 19:00:06.689969078 +0000 UTC m=+799.274452130"
Oct 30 19:00:08.451949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:08.451920    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 19:00:08.452363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:08.452117    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 19:00:08.593138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:08.593106    5232 generic.go:356] "Generic (PLEG): container finished" podID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerID="7ce1be1f83ae481c12e7e876519d39ec661d1619912e720c9781e3f62af44224" exitCode=60
Oct 30 19:00:08.593294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:08.593185    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" event={"ID":"d1bda426-2b67-4462-9e10-ae982167c9f3","Type":"ContainerDied","Data":"7ce1be1f83ae481c12e7e876519d39ec661d1619912e720c9781e3f62af44224"}
Oct 30 19:00:08.593457 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:08.593443    5232 scope.go:122] "RemoveContainer" containerID="7ce1be1f83ae481c12e7e876519d39ec661d1619912e720c9781e3f62af44224"
Oct 30 19:00:09.602074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:09.602037    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" event={"ID":"d1bda426-2b67-4462-9e10-ae982167c9f3","Type":"ContainerStarted","Data":"7640893911a767fdfea027ee2be221d6b6a4782336e083012982fd5b62ba838c"}
Oct 30 19:00:09.715851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:09.715773    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" podStartSLOduration=3.053685229 podStartE2EDuration="9.71574665s" totalImagesPullingTime="6.662061421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:09.713878555 +0000 UTC m=+802.298361553" watchObservedRunningTime="2026-10-30 19:00:09.71574665 +0000 UTC m=+802.300229674"
Oct 30 19:00:10.721524 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:10.721483    5232 generic.go:356] "Generic (PLEG): container finished" podID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerID="7640893911a767fdfea027ee2be221d6b6a4782336e083012982fd5b62ba838c" exitCode=60
Oct 30 19:00:10.722223 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:10.722196    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" event={"ID":"d1bda426-2b67-4462-9e10-ae982167c9f3","Type":"ContainerDied","Data":"7640893911a767fdfea027ee2be221d6b6a4782336e083012982fd5b62ba838c"}
Oct 30 19:00:10.723179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:10.722848    5232 scope.go:122] "RemoveContainer" containerID="7640893911a767fdfea027ee2be221d6b6a4782336e083012982fd5b62ba838c"
Oct 30 19:00:10.723613 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:10.723589    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889780-xr6qv_odh-ai-gateway-infra(d1bda426-2b67-4462-9e10-ae982167c9f3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3"
Oct 30 19:00:10.723972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:10.723954    5232 scope.go:122] "RemoveContainer" containerID="7ce1be1f83ae481c12e7e876519d39ec661d1619912e720c9781e3f62af44224"
Oct 30 19:00:11.484379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:11.484343    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 19:00:11.484549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:11.484535    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 19:00:11.765898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:11.733110    5232 scope.go:122] "RemoveContainer" containerID="7640893911a767fdfea027ee2be221d6b6a4782336e083012982fd5b62ba838c"
Oct 30 19:00:11.765898 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:11.733468    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889780-xr6qv_odh-ai-gateway-infra(d1bda426-2b67-4462-9e10-ae982167c9f3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3"
Oct 30 19:00:12.588132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:12.588094    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:12.765732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:12.765701    5232 generic.go:356] "Generic (PLEG): container finished" podID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerID="eaef2c5d17fbcc30ca48ef21bbdf448e012ffc7952b2512ea929725f2e2ea3e7" exitCode=0
Oct 30 19:00:12.765944 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:12.765779    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" event={"ID":"7c5e8235-3b0b-4684-b9c8-7438399620ab","Type":"ContainerDied","Data":"eaef2c5d17fbcc30ca48ef21bbdf448e012ffc7952b2512ea929725f2e2ea3e7"}
Oct 30 19:00:12.766613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:12.766596    5232 scope.go:122] "RemoveContainer" containerID="eaef2c5d17fbcc30ca48ef21bbdf448e012ffc7952b2512ea929725f2e2ea3e7"
Oct 30 19:00:12.767995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:12.767967    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:12.768078 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:12.768034    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:12.768078 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:12.768067    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:13.793260 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:13.793223    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" event={"ID":"7c5e8235-3b0b-4684-b9c8-7438399620ab","Type":"ContainerStarted","Data":"9a3d8f215c2dda4523205e7b1548a8b821331acb7e0641fcd48180bbe101c70f"}
Oct 30 19:00:14.136346 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:14.136314    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 19:00:14.161059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:14.161019    5232 patch_prober.go:29] interesting pod/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:00:14.161338 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:14.161310    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:00:15.136917 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:15.136879    5232 patch_prober.go:29] interesting pod/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:00:15.137621 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:15.137590    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:00:16.154235 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:16.154196    5232 patch_prober.go:29] interesting pod/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:00:16.154870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:16.154798    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:00:16.206023 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:16.205976    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:00:16.375967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:16.375922    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 19:00:17.136273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:17.136236    5232 patch_prober.go:29] interesting pod/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:00:17.136463 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:17.136302    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:00:18.145847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:18.136901    5232 patch_prober.go:29] interesting pod/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:00:18.145847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:18.136963    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:00:18.422098 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:18.421998    5232 patch_prober.go:29] interesting pod/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:18.422098 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:18.422069    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:19.141726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:19.141694    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 19:00:19.142851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:19.142830    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 19:00:20.851181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:20.851151    5232 generic.go:356] "Generic (PLEG): container finished" podID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerID="ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675" exitCode=0
Oct 30 19:00:20.851619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:20.851218    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" event={"ID":"a09fbe39-3d63-4b7e-843f-83bfabd433fb","Type":"ContainerDied","Data":"ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675"}
Oct 30 19:00:20.851619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:20.851606    5232 scope.go:122] "RemoveContainer" containerID="ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675"
Oct 30 19:00:20.852804 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:20.852774    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:20.852926 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:20.852872    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:20.852989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:20.852924    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:21.869432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:21.869396    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" event={"ID":"a09fbe39-3d63-4b7e-843f-83bfabd433fb","Type":"ContainerStarted","Data":"492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193"}
Oct 30 19:00:22.532882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:22.532852    5232 scope.go:122] "RemoveContainer" containerID="7640893911a767fdfea027ee2be221d6b6a4782336e083012982fd5b62ba838c"
Oct 30 19:00:22.814979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:22.814946    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 19:00:22.815151 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:22.815125    5232 patch_prober.go:29] interesting pod/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.815234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:22.815173    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 19:00:23.629588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.629552    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"]
Oct 30 19:00:23.633152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.633130    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.683916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.683874    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-df563ff7-serving-cert\""
Oct 30 19:00:23.712066 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.712026    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmxjv\" (UniqueName: \"kubernetes.io/projected/088bb2ad-1f04-4af8-825f-2114a17e85cf-kube-api-access-bmxjv\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.712217 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.712140    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-metrics-certs\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.712308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.712286    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/088bb2ad-1f04-4af8-825f-2114a17e85cf-openshift-service-ca\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.712349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.712328    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.735209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.735178    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"]
Oct 30 19:00:23.812905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.812801    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/088bb2ad-1f04-4af8-825f-2114a17e85cf-openshift-service-ca\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.812905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.812859    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.812905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.812886    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmxjv\" (UniqueName: \"kubernetes.io/projected/088bb2ad-1f04-4af8-825f-2114a17e85cf-kube-api-access-bmxjv\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.813190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.812913    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-metrics-certs\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.813190 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:23.813082    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-serving-cert: secret "maas-api-e2e-ait-df563ff7-serving-cert" not found
Oct 30 19:00:23.813190 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:23.813149    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls podName:088bb2ad-1f04-4af8-825f-2114a17e85cf nodeName:}" failed. No retries permitted until 2026-10-30 19:00:24.313129869 +0000 UTC m=+816.897612852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls") pod "maas-api-e2e-ait-df563ff7-586dd7df89-njpkf" (UID: "088bb2ad-1f04-4af8-825f-2114a17e85cf") : secret "maas-api-e2e-ait-df563ff7-serving-cert" not found
Oct 30 19:00:23.813582 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.813555    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/088bb2ad-1f04-4af8-825f-2114a17e85cf-openshift-service-ca\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.815200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.815172    5232 patch_prober.go:29] interesting pod/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:00:23.815295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.815238    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 19:00:23.815352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.815318    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-metrics-certs\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.872667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.872632    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmxjv\" (UniqueName: \"kubernetes.io/projected/088bb2ad-1f04-4af8-825f-2114a17e85cf-kube-api-access-bmxjv\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:23.883402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.883369    5232 generic.go:356] "Generic (PLEG): container finished" podID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerID="8e75f3d4ed8fc1c7a3dca6f96d0b6479787d2df11f0880cdafe5cc6c6bd8986e" exitCode=60
Oct 30 19:00:23.883549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.883433    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" event={"ID":"d1bda426-2b67-4462-9e10-ae982167c9f3","Type":"ContainerDied","Data":"8e75f3d4ed8fc1c7a3dca6f96d0b6479787d2df11f0880cdafe5cc6c6bd8986e"}
Oct 30 19:00:23.883549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.883498    5232 scope.go:122] "RemoveContainer" containerID="7640893911a767fdfea027ee2be221d6b6a4782336e083012982fd5b62ba838c"
Oct 30 19:00:23.883852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:23.883826    5232 scope.go:122] "RemoveContainer" containerID="8e75f3d4ed8fc1c7a3dca6f96d0b6479787d2df11f0880cdafe5cc6c6bd8986e"
Oct 30 19:00:23.884090 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:23.884074    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889780-xr6qv_odh-ai-gateway-infra(d1bda426-2b67-4462-9e10-ae982167c9f3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3"
Oct 30 19:00:24.317637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:24.317600    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:24.320109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:24.320083    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls\") pod \"maas-api-e2e-ait-df563ff7-586dd7df89-njpkf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:24.606176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:24.606149    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:24.815603 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:24.815568    5232 patch_prober.go:29] interesting pod/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:00:24.816133 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:24.815624    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 19:00:24.998460 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:24.998375    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"]
Oct 30 19:00:25.003463 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:00:25.003431    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088bb2ad_1f04_4af8_825f_2114a17e85cf.slice/crio-fb70019bd82a47fd7536fc31e7b7a0c988e63295c81f84119e0ca4e2a4777ce1 WatchSource:0}: Error finding container fb70019bd82a47fd7536fc31e7b7a0c988e63295c81f84119e0ca4e2a4777ce1: Status 404 returned error can't find the container with id fb70019bd82a47fd7536fc31e7b7a0c988e63295c81f84119e0ca4e2a4777ce1
Oct 30 19:00:25.062877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.062834    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"]
Oct 30 19:00:25.222912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.222889    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:25.326609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.326526    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1bda426-2b67-4462-9e10-ae982167c9f3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bda426-2b67-4462-9e10-ae982167c9f3-maas-api-service-ca\") pod \"d1bda426-2b67-4462-9e10-ae982167c9f3\" (UID: \"d1bda426-2b67-4462-9e10-ae982167c9f3\") "
Oct 30 19:00:25.326609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.326590    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1bda426-2b67-4462-9e10-ae982167c9f3-kube-api-access-lvgdb\" (UniqueName: \"kubernetes.io/projected/d1bda426-2b67-4462-9e10-ae982167c9f3-kube-api-access-lvgdb\") pod \"d1bda426-2b67-4462-9e10-ae982167c9f3\" (UID: \"d1bda426-2b67-4462-9e10-ae982167c9f3\") "
Oct 30 19:00:25.326964 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.326942    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1bda426-2b67-4462-9e10-ae982167c9f3-maas-api-service-ca" pod "d1bda426-2b67-4462-9e10-ae982167c9f3" (UID: "d1bda426-2b67-4462-9e10-ae982167c9f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:25.328673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.328647    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1bda426-2b67-4462-9e10-ae982167c9f3-kube-api-access-lvgdb" pod "d1bda426-2b67-4462-9e10-ae982167c9f3" (UID: "d1bda426-2b67-4462-9e10-ae982167c9f3"). InnerVolumeSpecName "kube-api-access-lvgdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:25.428273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.428243    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bda426-2b67-4462-9e10-ae982167c9f3-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:25.428273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.428271    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvgdb\" (UniqueName: \"kubernetes.io/projected/d1bda426-2b67-4462-9e10-ae982167c9f3-kube-api-access-lvgdb\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:25.815740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.815707    5232 patch_prober.go:29] interesting pod/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:00:25.816143 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.815762    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 19:00:25.897263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.897220    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"
Oct 30 19:00:25.897473 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.897261    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv" event={"ID":"d1bda426-2b67-4462-9e10-ae982167c9f3","Type":"ContainerDied","Data":"09301615b5618ea05e8aa183372c2a8ae4c31c7f709dafbe427f2836a5c63a8a"}
Oct 30 19:00:25.897473 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.897329    5232 scope.go:122] "RemoveContainer" containerID="8e75f3d4ed8fc1c7a3dca6f96d0b6479787d2df11f0880cdafe5cc6c6bd8986e"
Oct 30 19:00:25.898877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:25.898800    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf" event={"ID":"088bb2ad-1f04-4af8-825f-2114a17e85cf","Type":"ContainerStarted","Data":"fb70019bd82a47fd7536fc31e7b7a0c988e63295c81f84119e0ca4e2a4777ce1"}
Oct 30 19:00:26.063370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:26.063332    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"]
Oct 30 19:00:26.065420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:26.065398    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889780-xr6qv"]
Oct 30 19:00:26.815073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:26.814992    5232 patch_prober.go:29] interesting pod/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:00:26.815073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:26.815056    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 19:00:26.909524 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:26.909474    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf" event={"ID":"088bb2ad-1f04-4af8-825f-2114a17e85cf","Type":"ContainerStarted","Data":"4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051"}
Oct 30 19:00:26.910050 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:26.909597    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:27.031619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:27.031567    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf" podStartSLOduration=3.126610166 podStartE2EDuration="4.031555411s" totalImagesPullingTime="904.945245ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:27.031424171 +0000 UTC m=+819.615907171" watchObservedRunningTime="2026-10-30 19:00:27.031555411 +0000 UTC m=+819.616038407"
Oct 30 19:00:27.539129 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:27.539096    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" path="/var/lib/kubelet/pods/d1bda426-2b67-4462-9e10-ae982167c9f3/volumes"
Oct 30 19:00:27.819140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:27.819054    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 19:00:27.819499 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:27.819479    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 19:00:27.820185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:27.820168    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 19:00:32.922114 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:32.922082    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:00:46.188790 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:46.188752    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:00:49.934998 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:49.934934    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"]
Oct 30 19:00:50.036518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:50.036488    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"]
Oct 30 19:00:52.435239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:52.435205    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"]
Oct 30 19:00:54.935893 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:54.935826    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj" podUID="65caf4dd-f404-4155-af77-5a1b8462fe2d" containerName="payload-processing" containerID="cri-o://e65c5eb6f71786ca7b8637cc389b1aeb9b3dccb9e06c0c6abf418cb004a82f3f" gracePeriod=25
Oct 30 19:00:55.036955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.036872    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c" podUID="e2b47854-adb1-4f64-ab2d-7aab1267570b" containerName="payload-pre-processing" containerID="cri-o://24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a" gracePeriod=25
Oct 30 19:00:55.073746 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.073713    5232 generic.go:356] "Generic (PLEG): container finished" podID="65caf4dd-f404-4155-af77-5a1b8462fe2d" containerID="e65c5eb6f71786ca7b8637cc389b1aeb9b3dccb9e06c0c6abf418cb004a82f3f" exitCode=0
Oct 30 19:00:55.073929 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.073785    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj" event={"ID":"65caf4dd-f404-4155-af77-5a1b8462fe2d","Type":"ContainerDied","Data":"e65c5eb6f71786ca7b8637cc389b1aeb9b3dccb9e06c0c6abf418cb004a82f3f"}
Oct 30 19:00:55.191793 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.191722    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 19:00:55.239447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.239412    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65caf4dd-f404-4155-af77-5a1b8462fe2d-kube-api-access-dhkvn\" (UniqueName: \"kubernetes.io/projected/65caf4dd-f404-4155-af77-5a1b8462fe2d-kube-api-access-dhkvn\") pod \"65caf4dd-f404-4155-af77-5a1b8462fe2d\" (UID: \"65caf4dd-f404-4155-af77-5a1b8462fe2d\") "
Oct 30 19:00:55.239631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.239611    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65caf4dd-f404-4155-af77-5a1b8462fe2d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65caf4dd-f404-4155-af77-5a1b8462fe2d-plugins-config-volume\") pod \"65caf4dd-f404-4155-af77-5a1b8462fe2d\" (UID: \"65caf4dd-f404-4155-af77-5a1b8462fe2d\") "
Oct 30 19:00:55.239995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.239972    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65caf4dd-f404-4155-af77-5a1b8462fe2d-plugins-config-volume" pod "65caf4dd-f404-4155-af77-5a1b8462fe2d" (UID: "65caf4dd-f404-4155-af77-5a1b8462fe2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:55.241734 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.241705    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65caf4dd-f404-4155-af77-5a1b8462fe2d-kube-api-access-dhkvn" pod "65caf4dd-f404-4155-af77-5a1b8462fe2d" (UID: "65caf4dd-f404-4155-af77-5a1b8462fe2d"). InnerVolumeSpecName "kube-api-access-dhkvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:55.277801 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.277776    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 19:00:55.340484 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.340449    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b47854-adb1-4f64-ab2d-7aab1267570b-kube-api-access-pmkp4\" (UniqueName: \"kubernetes.io/projected/e2b47854-adb1-4f64-ab2d-7aab1267570b-kube-api-access-pmkp4\") pod \"e2b47854-adb1-4f64-ab2d-7aab1267570b\" (UID: \"e2b47854-adb1-4f64-ab2d-7aab1267570b\") "
Oct 30 19:00:55.340673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.340520    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2b47854-adb1-4f64-ab2d-7aab1267570b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b47854-adb1-4f64-ab2d-7aab1267570b-plugins-config-volume\") pod \"e2b47854-adb1-4f64-ab2d-7aab1267570b\" (UID: \"e2b47854-adb1-4f64-ab2d-7aab1267570b\") "
Oct 30 19:00:55.340852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.340828    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b47854-adb1-4f64-ab2d-7aab1267570b-plugins-config-volume" pod "e2b47854-adb1-4f64-ab2d-7aab1267570b" (UID: "e2b47854-adb1-4f64-ab2d-7aab1267570b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:55.340927 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.340892    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65caf4dd-f404-4155-af77-5a1b8462fe2d-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:55.340927 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.340913    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhkvn\" (UniqueName: \"kubernetes.io/projected/65caf4dd-f404-4155-af77-5a1b8462fe2d-kube-api-access-dhkvn\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:55.342592 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.342569    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b47854-adb1-4f64-ab2d-7aab1267570b-kube-api-access-pmkp4" pod "e2b47854-adb1-4f64-ab2d-7aab1267570b" (UID: "e2b47854-adb1-4f64-ab2d-7aab1267570b"). InnerVolumeSpecName "kube-api-access-pmkp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:55.441479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.441445    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b47854-adb1-4f64-ab2d-7aab1267570b-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:55.441479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.441476    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmkp4\" (UniqueName: \"kubernetes.io/projected/e2b47854-adb1-4f64-ab2d-7aab1267570b-kube-api-access-pmkp4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:55.825123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.825043    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"]
Oct 30 19:00:55.922874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:55.922835    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"]
Oct 30 19:00:56.081087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.080993    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj" event={"ID":"65caf4dd-f404-4155-af77-5a1b8462fe2d","Type":"ContainerDied","Data":"4f73ce15cd781c851f2a0d0751e04578f7442454b548d50c9f2fd703f1356eef"}
Oct 30 19:00:56.081087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.081048    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"
Oct 30 19:00:56.081087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.081078    5232 scope.go:122] "RemoveContainer" containerID="e65c5eb6f71786ca7b8637cc389b1aeb9b3dccb9e06c0c6abf418cb004a82f3f"
Oct 30 19:00:56.082484 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.082456    5232 generic.go:356] "Generic (PLEG): container finished" podID="e2b47854-adb1-4f64-ab2d-7aab1267570b" containerID="24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a" exitCode=0
Oct 30 19:00:56.082598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.082509    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"
Oct 30 19:00:56.082598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.082551    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c" event={"ID":"e2b47854-adb1-4f64-ab2d-7aab1267570b","Type":"ContainerDied","Data":"24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a"}
Oct 30 19:00:56.082720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.082595    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c" event={"ID":"e2b47854-adb1-4f64-ab2d-7aab1267570b","Type":"ContainerDied","Data":"15610cccbfe7592d1a548f658702d7c93c011795b404e8c7561ad5de22c9f5b4"}
Oct 30 19:00:56.091912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.091889    5232 scope.go:122] "RemoveContainer" containerID="24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a"
Oct 30 19:00:56.104337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.104314    5232 scope.go:122] "RemoveContainer" containerID="24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a"
Oct 30 19:00:56.104708 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:56.104685    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a\": container with ID starting with 24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a not found: ID does not exist" containerID="24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a"
Oct 30 19:00:56.104787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.104719    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a"} err="failed to get container status \"24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a\": rpc error: code = NotFound desc = could not find container \"24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a\": container with ID starting with 24c9d9f6be54933b7d0d192824b24be7c2487a664c92d448b09559d11a8c938a not found: ID does not exist"
Oct 30 19:00:56.239737 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.239703    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"]
Oct 30 19:00:56.243614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.243584    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e7b96f88-7674bb66d-j95gj"]
Oct 30 19:00:56.348770 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.348733    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"]
Oct 30 19:00:56.353221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:56.353193    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e7b96f88-85f7588ff-snc9c"]
Oct 30 19:00:57.436244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.436202    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds" podUID="3eab5908-871d-4de6-a981-8ac079ebe40b" containerName="payload-pre-processing" containerID="cri-o://25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666" gracePeriod=25
Oct 30 19:00:57.552838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.552523    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65caf4dd-f404-4155-af77-5a1b8462fe2d" path="/var/lib/kubelet/pods/65caf4dd-f404-4155-af77-5a1b8462fe2d/volumes"
Oct 30 19:00:57.557857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.553201    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2b47854-adb1-4f64-ab2d-7aab1267570b" path="/var/lib/kubelet/pods/e2b47854-adb1-4f64-ab2d-7aab1267570b/volumes"
Oct 30 19:00:57.699299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.699218    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 19:00:57.761822 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.761781    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3eab5908-871d-4de6-a981-8ac079ebe40b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eab5908-871d-4de6-a981-8ac079ebe40b-plugins-config-volume\") pod \"3eab5908-871d-4de6-a981-8ac079ebe40b\" (UID: \"3eab5908-871d-4de6-a981-8ac079ebe40b\") "
Oct 30 19:00:57.762009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.761832    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eab5908-871d-4de6-a981-8ac079ebe40b-kube-api-access-hl2vc\" (UniqueName: \"kubernetes.io/projected/3eab5908-871d-4de6-a981-8ac079ebe40b-kube-api-access-hl2vc\") pod \"3eab5908-871d-4de6-a981-8ac079ebe40b\" (UID: \"3eab5908-871d-4de6-a981-8ac079ebe40b\") "
Oct 30 19:00:57.762237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.762218    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eab5908-871d-4de6-a981-8ac079ebe40b-plugins-config-volume" pod "3eab5908-871d-4de6-a981-8ac079ebe40b" (UID: "3eab5908-871d-4de6-a981-8ac079ebe40b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:57.764080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.764053    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eab5908-871d-4de6-a981-8ac079ebe40b-kube-api-access-hl2vc" pod "3eab5908-871d-4de6-a981-8ac079ebe40b" (UID: "3eab5908-871d-4de6-a981-8ac079ebe40b"). InnerVolumeSpecName "kube-api-access-hl2vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:57.862800 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.862766    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eab5908-871d-4de6-a981-8ac079ebe40b-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:57.862800 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:57.862797    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl2vc\" (UniqueName: \"kubernetes.io/projected/3eab5908-871d-4de6-a981-8ac079ebe40b-kube-api-access-hl2vc\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:58.097849 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.097793    5232 generic.go:356] "Generic (PLEG): container finished" podID="3eab5908-871d-4de6-a981-8ac079ebe40b" containerID="25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666" exitCode=0
Oct 30 19:00:58.098014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.097861    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds" event={"ID":"3eab5908-871d-4de6-a981-8ac079ebe40b","Type":"ContainerDied","Data":"25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666"}
Oct 30 19:00:58.098014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.097873    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"
Oct 30 19:00:58.098014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.097911    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds" event={"ID":"3eab5908-871d-4de6-a981-8ac079ebe40b","Type":"ContainerDied","Data":"e25ccf21a75e3a4bfd028d4425141412e0a18340dccc4c0d0f379561de26d6d8"}
Oct 30 19:00:58.098014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.097933    5232 scope.go:122] "RemoveContainer" containerID="25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666"
Oct 30 19:00:58.107672 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.107647    5232 scope.go:122] "RemoveContainer" containerID="25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666"
Oct 30 19:00:58.107994 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:58.107975    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666\": container with ID starting with 25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666 not found: ID does not exist" containerID="25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666"
Oct 30 19:00:58.108057 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.108004    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666"} err="failed to get container status \"25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666\": rpc error: code = NotFound desc = could not find container \"25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666\": container with ID starting with 25d135f5bfa12d4ca961c480617643dce7ed2ed4fe7f9608983327a12ce1c666 not found: ID does not exist"
Oct 30 19:00:58.257662 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.257625    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"]
Oct 30 19:00:58.305181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.305143    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-45e6849e-dbcb6b659-2ttds"]
Oct 30 19:00:58.664228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.664193    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"]
Oct 30 19:00:58.664713 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.664462    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc" podUID="b4a17886-d3bb-4b31-a671-bea255803eaa" containerName="maas-api" containerID="cri-o://0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4" gracePeriod=30
Oct 30 19:00:58.921451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.921383    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:58.971930 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.971898    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4a17886-d3bb-4b31-a671-bea255803eaa-kube-api-access-rc6cf\" (UniqueName: \"kubernetes.io/projected/b4a17886-d3bb-4b31-a671-bea255803eaa-kube-api-access-rc6cf\") pod \"b4a17886-d3bb-4b31-a671-bea255803eaa\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") "
Oct 30 19:00:58.972110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.971971    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b4a17886-d3bb-4b31-a671-bea255803eaa-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a17886-d3bb-4b31-a671-bea255803eaa-openshift-service-ca\") pod \"b4a17886-d3bb-4b31-a671-bea255803eaa\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") "
Oct 30 19:00:58.972110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.972015    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls\") pod \"b4a17886-d3bb-4b31-a671-bea255803eaa\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") "
Oct 30 19:00:58.972110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.972043    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-metrics-certs\") pod \"b4a17886-d3bb-4b31-a671-bea255803eaa\" (UID: \"b4a17886-d3bb-4b31-a671-bea255803eaa\") "
Oct 30 19:00:58.972364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.972331    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4a17886-d3bb-4b31-a671-bea255803eaa-openshift-service-ca" pod "b4a17886-d3bb-4b31-a671-bea255803eaa" (UID: "b4a17886-d3bb-4b31-a671-bea255803eaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:58.972498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.972427    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a17886-d3bb-4b31-a671-bea255803eaa-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:58.974184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.974156    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls" pod "b4a17886-d3bb-4b31-a671-bea255803eaa" (UID: "b4a17886-d3bb-4b31-a671-bea255803eaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:58.974184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.974171    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-metrics-certs" pod "b4a17886-d3bb-4b31-a671-bea255803eaa" (UID: "b4a17886-d3bb-4b31-a671-bea255803eaa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:58.974315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:58.974297    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a17886-d3bb-4b31-a671-bea255803eaa-kube-api-access-rc6cf" pod "b4a17886-d3bb-4b31-a671-bea255803eaa" (UID: "b4a17886-d3bb-4b31-a671-bea255803eaa"). InnerVolumeSpecName "kube-api-access-rc6cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:59.073161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.073121    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc6cf\" (UniqueName: \"kubernetes.io/projected/b4a17886-d3bb-4b31-a671-bea255803eaa-kube-api-access-rc6cf\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:59.073161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.073156    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:59.073161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.073166    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4a17886-d3bb-4b31-a671-bea255803eaa-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:59.105343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.105305    5232 generic.go:356] "Generic (PLEG): container finished" podID="b4a17886-d3bb-4b31-a671-bea255803eaa" containerID="0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4" exitCode=0
Oct 30 19:00:59.105511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.105365    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"
Oct 30 19:00:59.105511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.105386    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc" event={"ID":"b4a17886-d3bb-4b31-a671-bea255803eaa","Type":"ContainerDied","Data":"0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4"}
Oct 30 19:00:59.105511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.105458    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc" event={"ID":"b4a17886-d3bb-4b31-a671-bea255803eaa","Type":"ContainerDied","Data":"c86b6986a40a5f013e7056968594f9d1f0685c8bf2ce7792cff7c5b404119a02"}
Oct 30 19:00:59.105511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.105486    5232 scope.go:122] "RemoveContainer" containerID="0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4"
Oct 30 19:00:59.115159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.115136    5232 scope.go:122] "RemoveContainer" containerID="0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4"
Oct 30 19:00:59.115420 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:00:59.115400    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4\": container with ID starting with 0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4 not found: ID does not exist" containerID="0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4"
Oct 30 19:00:59.115465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.115430    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4"} err="failed to get container status \"0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4\": rpc error: code = NotFound desc = could not find container \"0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4\": container with ID starting with 0cd7d823d06e9456291e68ec2b9848a169dd29016488fdfa04eba1645ed40db4 not found: ID does not exist"
Oct 30 19:00:59.265208 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.265118    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"]
Oct 30 19:00:59.271793 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.271759    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b4e8ca3e-b8c646749-znpgc"]
Oct 30 19:00:59.539282 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.539195    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3eab5908-871d-4de6-a981-8ac079ebe40b" path="/var/lib/kubelet/pods/3eab5908-871d-4de6-a981-8ac079ebe40b/volumes"
Oct 30 19:00:59.539709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.539688    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4a17886-d3bb-4b31-a671-bea255803eaa" path="/var/lib/kubelet/pods/b4a17886-d3bb-4b31-a671-bea255803eaa/volumes"
Oct 30 19:00:59.785895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.785859    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"]
Oct 30 19:00:59.786336 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:00:59.786167    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy" containerID="cri-o://492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193" gracePeriod=30
Oct 30 19:01:00.825388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:00.825350    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5" podUID="b5896ae3-0199-4a33-8818-ef2317ab1c43" containerName="payload-processing" containerID="cri-o://11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49" gracePeriod=25
Oct 30 19:01:00.923732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:00.923658    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq" podUID="b4a5b01f-5be8-49aa-834f-c762a0ac8583" containerName="payload-pre-processing" containerID="cri-o://3fd9a0fbd703b021ef183ab6e312ec51f3d501c15e4343ce1742307197bddd22" gracePeriod=25
Oct 30 19:01:01.076450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.076388    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 19:01:01.096295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.096269    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5896ae3-0199-4a33-8818-ef2317ab1c43-kube-api-access-kfmg7\" (UniqueName: \"kubernetes.io/projected/b5896ae3-0199-4a33-8818-ef2317ab1c43-kube-api-access-kfmg7\") pod \"b5896ae3-0199-4a33-8818-ef2317ab1c43\" (UID: \"b5896ae3-0199-4a33-8818-ef2317ab1c43\") "
Oct 30 19:01:01.096485 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.096469    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5896ae3-0199-4a33-8818-ef2317ab1c43-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5896ae3-0199-4a33-8818-ef2317ab1c43-plugins-config-volume\") pod \"b5896ae3-0199-4a33-8818-ef2317ab1c43\" (UID: \"b5896ae3-0199-4a33-8818-ef2317ab1c43\") "
Oct 30 19:01:01.097055 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.097030    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5896ae3-0199-4a33-8818-ef2317ab1c43-plugins-config-volume" pod "b5896ae3-0199-4a33-8818-ef2317ab1c43" (UID: "b5896ae3-0199-4a33-8818-ef2317ab1c43"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:01.101387 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.101359    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5896ae3-0199-4a33-8818-ef2317ab1c43-kube-api-access-kfmg7" pod "b5896ae3-0199-4a33-8818-ef2317ab1c43" (UID: "b5896ae3-0199-4a33-8818-ef2317ab1c43"). InnerVolumeSpecName "kube-api-access-kfmg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:01.121996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.121968    5232 generic.go:356] "Generic (PLEG): container finished" podID="b5896ae3-0199-4a33-8818-ef2317ab1c43" containerID="11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49" exitCode=0
Oct 30 19:01:01.122127 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.122089    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"
Oct 30 19:01:01.122127 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.122098    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5" event={"ID":"b5896ae3-0199-4a33-8818-ef2317ab1c43","Type":"ContainerDied","Data":"11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49"}
Oct 30 19:01:01.122235 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.122158    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5" event={"ID":"b5896ae3-0199-4a33-8818-ef2317ab1c43","Type":"ContainerDied","Data":"97f2d14a46ca299f1cb1c72e4369643dc2db35bcbd26ff99d6cf45468856fc3b"}
Oct 30 19:01:01.122235 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.122191    5232 scope.go:122] "RemoveContainer" containerID="11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49"
Oct 30 19:01:01.123856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.123830    5232 generic.go:356] "Generic (PLEG): container finished" podID="b4a5b01f-5be8-49aa-834f-c762a0ac8583" containerID="3fd9a0fbd703b021ef183ab6e312ec51f3d501c15e4343ce1742307197bddd22" exitCode=0
Oct 30 19:01:01.123973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.123881    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq" event={"ID":"b4a5b01f-5be8-49aa-834f-c762a0ac8583","Type":"ContainerDied","Data":"3fd9a0fbd703b021ef183ab6e312ec51f3d501c15e4343ce1742307197bddd22"}
Oct 30 19:01:01.133492 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.133467    5232 scope.go:122] "RemoveContainer" containerID="11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49"
Oct 30 19:01:01.133873 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:01.133845    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49\": container with ID starting with 11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49 not found: ID does not exist" containerID="11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49"
Oct 30 19:01:01.133985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.133886    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49"} err="failed to get container status \"11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49\": rpc error: code = NotFound desc = could not find container \"11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49\": container with ID starting with 11d5617429f15141ee8b429fa1652cdf55b523eacd66b2306ad04c91a162ae49 not found: ID does not exist"
Oct 30 19:01:01.178345 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.178320    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 19:01:01.197718 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.197687    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4a5b01f-5be8-49aa-834f-c762a0ac8583-kube-api-access-wbh7b\" (UniqueName: \"kubernetes.io/projected/b4a5b01f-5be8-49aa-834f-c762a0ac8583-kube-api-access-wbh7b\") pod \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\" (UID: \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\") "
Oct 30 19:01:01.197870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.197738    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b4a5b01f-5be8-49aa-834f-c762a0ac8583-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4a5b01f-5be8-49aa-834f-c762a0ac8583-plugins-config-volume\") pod \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\" (UID: \"b4a5b01f-5be8-49aa-834f-c762a0ac8583\") "
Oct 30 19:01:01.198161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.198089    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5896ae3-0199-4a33-8818-ef2317ab1c43-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:01.198161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.198112    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfmg7\" (UniqueName: \"kubernetes.io/projected/b5896ae3-0199-4a33-8818-ef2317ab1c43-kube-api-access-kfmg7\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:01.198447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.198418    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4a5b01f-5be8-49aa-834f-c762a0ac8583-plugins-config-volume" pod "b4a5b01f-5be8-49aa-834f-c762a0ac8583" (UID: "b4a5b01f-5be8-49aa-834f-c762a0ac8583"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:01.201964 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.201936    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a5b01f-5be8-49aa-834f-c762a0ac8583-kube-api-access-wbh7b" pod "b4a5b01f-5be8-49aa-834f-c762a0ac8583" (UID: "b4a5b01f-5be8-49aa-834f-c762a0ac8583"). InnerVolumeSpecName "kube-api-access-wbh7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:01.299024 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.298992    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbh7b\" (UniqueName: \"kubernetes.io/projected/b4a5b01f-5be8-49aa-834f-c762a0ac8583-kube-api-access-wbh7b\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:01.299024 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.299018    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4a5b01f-5be8-49aa-834f-c762a0ac8583-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:01.422716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.422676    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"]
Oct 30 19:01:01.426379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.426354    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b4e8ca3e-79d47686f6-9vst5"]
Oct 30 19:01:01.539339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:01.539303    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5896ae3-0199-4a33-8818-ef2317ab1c43" path="/var/lib/kubelet/pods/b5896ae3-0199-4a33-8818-ef2317ab1c43/volumes"
Oct 30 19:01:02.131830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:02.131727    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"
Oct 30 19:01:02.131830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:02.131723    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq" event={"ID":"b4a5b01f-5be8-49aa-834f-c762a0ac8583","Type":"ContainerDied","Data":"9c385700f44c4b7569deeecfdbe7ff9f9112e1a67a164172452725a5b0c816e6"}
Oct 30 19:01:02.132296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:02.131879    5232 scope.go:122] "RemoveContainer" containerID="3fd9a0fbd703b021ef183ab6e312ec51f3d501c15e4343ce1742307197bddd22"
Oct 30 19:01:02.291367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:02.291331    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"]
Oct 30 19:01:02.339067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:02.339034    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b4e8ca3e-98d96f795-w8mnq"]
Oct 30 19:01:03.540182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:03.540144    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4a5b01f-5be8-49aa-834f-c762a0ac8583" path="/var/lib/kubelet/pods/b4a5b01f-5be8-49aa-834f-c762a0ac8583/volumes"
Oct 30 19:01:05.040768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.040739    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 19:01:05.132158 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132128    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-kube-api-access-lvw2v\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-kube-api-access-lvw2v\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132184    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-socket\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132226    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istiod-ca-cert\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132258    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-credential-socket\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132290    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-podinfo\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132373    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-token\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-token\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132405    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-certs\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132501    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-envoy\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132526    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-socket" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:05.132769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132559    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-data\") pod \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\" (UID: \"a09fbe39-3d63-4b7e-843f-83bfabd433fb\") "
Oct 30 19:01:05.132769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132738    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-data" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:05.132892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132773    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istiod-ca-cert" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:05.133224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.132978    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.133224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.133005    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.133224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.133022    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.133224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.133004    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-certs" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:05.133224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.133157    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-credential-socket" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:05.134572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.134548    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-podinfo" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:01:05.134734 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.134716    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-token" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:05.134879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.134861    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-kube-api-access-lvw2v" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "kube-api-access-lvw2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:05.135040 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.135023    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-envoy" pod "a09fbe39-3d63-4b7e-843f-83bfabd433fb" (UID: "a09fbe39-3d63-4b7e-843f-83bfabd433fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:05.153026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.152999    5232 generic.go:356] "Generic (PLEG): container finished" podID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerID="492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193" exitCode=0
Oct 30 19:01:05.153153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.153050    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"
Oct 30 19:01:05.153153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.153075    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" event={"ID":"a09fbe39-3d63-4b7e-843f-83bfabd433fb","Type":"ContainerDied","Data":"492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193"}
Oct 30 19:01:05.153153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.153141    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn" event={"ID":"a09fbe39-3d63-4b7e-843f-83bfabd433fb","Type":"ContainerDied","Data":"9c95f5fad671bd3a84df0f17e491ddaab019b798614ee19775a2ea3e68c9fed5"}
Oct 30 19:01:05.153412 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.153170    5232 scope.go:122] "RemoveContainer" containerID="492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193"
Oct 30 19:01:05.162917 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.162894    5232 scope.go:122] "RemoveContainer" containerID="ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675"
Oct 30 19:01:05.171596 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.171578    5232 scope.go:122] "RemoveContainer" containerID="492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193"
Oct 30 19:01:05.171873 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:05.171853    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193\": container with ID starting with 492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193 not found: ID does not exist" containerID="492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193"
Oct 30 19:01:05.171946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.171887    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193"} err="failed to get container status \"492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193\": rpc error: code = NotFound desc = could not find container \"492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193\": container with ID starting with 492721fea8579e6161f12a0edea2c38088a1ecd9d9d4ae90bc633c0cead9f193 not found: ID does not exist"
Oct 30 19:01:05.171946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.171910    5232 scope.go:122] "RemoveContainer" containerID="ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675"
Oct 30 19:01:05.172160 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:05.172142    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675\": container with ID starting with ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675 not found: ID does not exist" containerID="ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675"
Oct 30 19:01:05.172197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.172169    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675"} err="failed to get container status \"ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675\": rpc error: code = NotFound desc = could not find container \"ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675\": container with ID starting with ded73ca9210ef5d4aade8460a2803d8d058b5f06bb44d4d400f4673b97851675 not found: ID does not exist"
Oct 30 19:01:05.233879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.233845    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.233879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.233873    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvw2v\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-kube-api-access-lvw2v\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.233879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.233886    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.234105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.233896    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.234105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.233904    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a09fbe39-3d63-4b7e-843f-83bfabd433fb-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.234105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.233913    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09fbe39-3d63-4b7e-843f-83bfabd433fb-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.312321 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.312284    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"]
Oct 30 19:01:05.316573 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.316544    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-45e6849e-gw-openshift-default-5574c88465-jvtpn"]
Oct 30 19:01:05.539070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.538989    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" path="/var/lib/kubelet/pods/a09fbe39-3d63-4b7e-843f-83bfabd433fb/volumes"
Oct 30 19:01:05.831025 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.830939    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"]
Oct 30 19:01:05.831206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:05.831158    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh" podUID="93321856-90e6-4fc5-96a5-5d879a432501" containerName="maas-api" containerID="cri-o://8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586" gracePeriod=30
Oct 30 19:01:06.091003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.090978    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 19:01:06.142060 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.142028    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93321856-90e6-4fc5-96a5-5d879a432501-kube-api-access-pkcvp\" (UniqueName: \"kubernetes.io/projected/93321856-90e6-4fc5-96a5-5d879a432501-kube-api-access-pkcvp\") pod \"93321856-90e6-4fc5-96a5-5d879a432501\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") "
Oct 30 19:01:06.142210 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.142064    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93321856-90e6-4fc5-96a5-5d879a432501-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93321856-90e6-4fc5-96a5-5d879a432501-openshift-service-ca\") pod \"93321856-90e6-4fc5-96a5-5d879a432501\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") "
Oct 30 19:01:06.142210 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.142116    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-metrics-certs\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-metrics-certs\") pod \"93321856-90e6-4fc5-96a5-5d879a432501\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") "
Oct 30 19:01:06.142359 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.142339    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-maas-api-tls\") pod \"93321856-90e6-4fc5-96a5-5d879a432501\" (UID: \"93321856-90e6-4fc5-96a5-5d879a432501\") "
Oct 30 19:01:06.142446 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.142420    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93321856-90e6-4fc5-96a5-5d879a432501-openshift-service-ca" pod "93321856-90e6-4fc5-96a5-5d879a432501" (UID: "93321856-90e6-4fc5-96a5-5d879a432501"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:06.142676 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.142661    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93321856-90e6-4fc5-96a5-5d879a432501-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:06.144090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.144068    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93321856-90e6-4fc5-96a5-5d879a432501-kube-api-access-pkcvp" pod "93321856-90e6-4fc5-96a5-5d879a432501" (UID: "93321856-90e6-4fc5-96a5-5d879a432501"). InnerVolumeSpecName "kube-api-access-pkcvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:06.144333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.144313    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-metrics-certs" pod "93321856-90e6-4fc5-96a5-5d879a432501" (UID: "93321856-90e6-4fc5-96a5-5d879a432501"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:06.144432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.144370    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-maas-api-tls" pod "93321856-90e6-4fc5-96a5-5d879a432501" (UID: "93321856-90e6-4fc5-96a5-5d879a432501"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:06.160769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.160746    5232 generic.go:356] "Generic (PLEG): container finished" podID="93321856-90e6-4fc5-96a5-5d879a432501" containerID="8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586" exitCode=0
Oct 30 19:01:06.160883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.160779    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh" event={"ID":"93321856-90e6-4fc5-96a5-5d879a432501","Type":"ContainerDied","Data":"8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586"}
Oct 30 19:01:06.160883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.160792    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"
Oct 30 19:01:06.160883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.160842    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh" event={"ID":"93321856-90e6-4fc5-96a5-5d879a432501","Type":"ContainerDied","Data":"266ff64cdd44c1dc9bc325a5db8413af5b9fe113e5d385e80730bb9c8fec4d94"}
Oct 30 19:01:06.160883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.160867    5232 scope.go:122] "RemoveContainer" containerID="8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586"
Oct 30 19:01:06.170330 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.170312    5232 scope.go:122] "RemoveContainer" containerID="8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586"
Oct 30 19:01:06.170610 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:06.170591    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586\": container with ID starting with 8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586 not found: ID does not exist" containerID="8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586"
Oct 30 19:01:06.170664 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.170619    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586"} err="failed to get container status \"8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586\": rpc error: code = NotFound desc = could not find container \"8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586\": container with ID starting with 8f0515a44ce826782b49a5e5fa4badec0ba39acb8980b3dd718333989e259586 not found: ID does not exist"
Oct 30 19:01:06.243338 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.243303    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkcvp\" (UniqueName: \"kubernetes.io/projected/93321856-90e6-4fc5-96a5-5d879a432501-kube-api-access-pkcvp\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:06.243338 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.243333    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:06.243338 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.243344    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93321856-90e6-4fc5-96a5-5d879a432501-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:06.312882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.312847    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"]
Oct 30 19:01:06.328367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:06.328326    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7b96f88-5b599d99c9-rx9hh"]
Oct 30 19:01:07.539048 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:07.539015    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93321856-90e6-4fc5-96a5-5d879a432501" path="/var/lib/kubelet/pods/93321856-90e6-4fc5-96a5-5d879a432501/volumes"
Oct 30 19:01:07.968162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:07.968131    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"]
Oct 30 19:01:07.968371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:07.968349    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy" containerID="cri-o://9a3d8f215c2dda4523205e7b1548a8b821331acb7e0641fcd48180bbe101c70f" gracePeriod=30
Oct 30 19:01:09.309300 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309266    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"]
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309638    5232 cpu_manager.go:414] "containerMap: removing container" podUID="b4a17886-d3bb-4b31-a671-bea255803eaa" containerName="maas-api"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309649    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4a17886-d3bb-4b31-a671-bea255803eaa" containerName="maas-api"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309658    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309663    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309679    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309684    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309690    5232 cpu_manager.go:414] "containerMap: removing container" podUID="3eab5908-871d-4de6-a981-8ac079ebe40b" containerName="payload-pre-processing"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309695    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3eab5908-871d-4de6-a981-8ac079ebe40b" containerName="payload-pre-processing"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309702    5232 cpu_manager.go:414] "containerMap: removing container" podUID="e2b47854-adb1-4f64-ab2d-7aab1267570b" containerName="payload-pre-processing"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309707    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b47854-adb1-4f64-ab2d-7aab1267570b" containerName="payload-pre-processing"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309714    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:09.309714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309718    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309731    5232 cpu_manager.go:414] "containerMap: removing container" podUID="b5896ae3-0199-4a33-8818-ef2317ab1c43" containerName="payload-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309736    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5896ae3-0199-4a33-8818-ef2317ab1c43" containerName="payload-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309745    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309749    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309757    5232 cpu_manager.go:414] "containerMap: removing container" podUID="93321856-90e6-4fc5-96a5-5d879a432501" containerName="maas-api"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309762    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93321856-90e6-4fc5-96a5-5d879a432501" containerName="maas-api"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309771    5232 cpu_manager.go:414] "containerMap: removing container" podUID="b4a5b01f-5be8-49aa-834f-c762a0ac8583" containerName="payload-pre-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309776    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4a5b01f-5be8-49aa-834f-c762a0ac8583" containerName="payload-pre-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309785    5232 cpu_manager.go:414] "containerMap: removing container" podUID="65caf4dd-f404-4155-af77-5a1b8462fe2d" containerName="payload-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309789    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65caf4dd-f404-4155-af77-5a1b8462fe2d" containerName="payload-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309871    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309878    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="3eab5908-871d-4de6-a981-8ac079ebe40b" containerName="payload-pre-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309884    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b47854-adb1-4f64-ab2d-7aab1267570b" containerName="payload-pre-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309891    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309899    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5896ae3-0199-4a33-8818-ef2317ab1c43" containerName="payload-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309904    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="93321856-90e6-4fc5-96a5-5d879a432501" containerName="maas-api"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309910    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4a5b01f-5be8-49aa-834f-c762a0ac8583" containerName="payload-pre-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309916    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="65caf4dd-f404-4155-af77-5a1b8462fe2d" containerName="payload-processing"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309921    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4a17886-d3bb-4b31-a671-bea255803eaa" containerName="maas-api"
Oct 30 19:01:09.310146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.309925    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:09.313680 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.313660    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.364871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.364832    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-330025-openshift-default-dockercfg-9qb6q\""
Oct 30 19:01:09.376600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376570    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20ab810-41dd-465a-8883-08f583b528ee-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.376757 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376616    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.376757 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376640    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.376871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376763    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.376871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376796    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.376871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376836    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.376987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376941    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.376987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376965    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pqcg\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-kube-api-access-4pqcg\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.377081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.376996    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20ab810-41dd-465a-8883-08f583b528ee-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.414065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.414034    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"]
Oct 30 19:01:09.478001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.477963    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20ab810-41dd-465a-8883-08f583b528ee-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478012    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478136    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478190    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478232    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478268    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478456 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478333    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478456 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478345    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478456 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478378    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pqcg\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-kube-api-access-4pqcg\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478456 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478445    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20ab810-41dd-465a-8883-08f583b528ee-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478651 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478571    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478651 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478592    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478651 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478642    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.478776 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.478751    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20ab810-41dd-465a-8883-08f583b528ee-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.480508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.480484    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.480918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.480897    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20ab810-41dd-465a-8883-08f583b528ee-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.537513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.537473    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.537694 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.537667    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pqcg\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-kube-api-access-4pqcg\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:09.626314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:09.626281    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:10.053134 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:10.053051    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"]
Oct 30 19:01:10.058114 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:01:10.058085    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20ab810_41dd_465a_8883_08f583b528ee.slice/crio-816e90bc9218dd8c8ed76c8badd53be33d0f50297587a3bbf86cf2a64d81a204 WatchSource:0}: Error finding container 816e90bc9218dd8c8ed76c8badd53be33d0f50297587a3bbf86cf2a64d81a204: Status 404 returned error can't find the container with id 816e90bc9218dd8c8ed76c8badd53be33d0f50297587a3bbf86cf2a64d81a204
Oct 30 19:01:10.060229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:10.060199    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:10.060307 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:10.060261    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:10.060307 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:10.060288    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:10.185752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:10.185717    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" event={"ID":"e20ab810-41dd-465a-8883-08f583b528ee","Type":"ContainerStarted","Data":"5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d"}
Oct 30 19:01:10.185939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:10.185775    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" event={"ID":"e20ab810-41dd-465a-8883-08f583b528ee","Type":"ContainerStarted","Data":"816e90bc9218dd8c8ed76c8badd53be33d0f50297587a3bbf86cf2a64d81a204"}
Oct 30 19:01:10.305669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:10.305552    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" podStartSLOduration=1.305537779 podStartE2EDuration="1.305537779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:10.304334219 +0000 UTC m=+862.888817218" watchObservedRunningTime="2026-10-30 19:01:10.305537779 +0000 UTC m=+862.890020776"
Oct 30 19:01:10.626436 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:10.626402    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:11.544771 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.544739    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"]
Oct 30 19:01:11.545146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.545134    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:11.545192 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.545148    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:11.545224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.545211    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1bda426-2b67-4462-9e10-ae982167c9f3" containerName="cleanup"
Oct 30 19:01:11.545224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.545220    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a09fbe39-3d63-4b7e-843f-83bfabd433fb" containerName="istio-proxy"
Oct 30 19:01:11.548371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.548348    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599138    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk9qk\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-kube-api-access-rk9qk\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599257    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599283    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599346    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599371    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5745ae03-a721-4681-80bc-369dc9932f96-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599460 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599408    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599492    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5745ae03-a721-4681-80bc-369dc9932f96-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599539    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.599581 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.599556    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.632202 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.632176    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:11.650382 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.650352    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"]
Oct 30 19:01:11.681642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.681593    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-22fb18-openshift-default-dockercfg-pqpjp\""
Oct 30 19:01:11.700837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.700793    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.700847    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.700886    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.700913    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5745ae03-a721-4681-80bc-369dc9932f96-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.700973    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701010    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5745ae03-a721-4681-80bc-369dc9932f96-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701037    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701058    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701093    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk9qk\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-kube-api-access-rk9qk\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701322 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701214    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701322 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701222    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701443    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701679    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.701853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.701836    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5745ae03-a721-4681-80bc-369dc9932f96-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.703436 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.703412    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.703776 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.703749    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5745ae03-a721-4681-80bc-369dc9932f96-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.763061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.763034    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.768033 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.768009    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk9qk\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-kube-api-access-rk9qk\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:11.861634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:11.861594    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:12.197408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:12.197326    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:12.198401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:12.198380    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:01:12.285190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:12.285158    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"]
Oct 30 19:01:12.290029 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:01:12.290003    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5745ae03_a721_4681_80bc_369dc9932f96.slice/crio-3e26dc413cb34c2b7f3a57578bb3ed6c4a99db3a56a235d643c6b7769e296f42 WatchSource:0}: Error finding container 3e26dc413cb34c2b7f3a57578bb3ed6c4a99db3a56a235d643c6b7769e296f42: Status 404 returned error can't find the container with id 3e26dc413cb34c2b7f3a57578bb3ed6c4a99db3a56a235d643c6b7769e296f42
Oct 30 19:01:12.292155 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:12.292120    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:12.292230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:12.292183    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:12.292230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:12.292214    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:13.203886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.203852    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" event={"ID":"5745ae03-a721-4681-80bc-369dc9932f96","Type":"ContainerStarted","Data":"9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19"}
Oct 30 19:01:13.204248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.203923    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" event={"ID":"5745ae03-a721-4681-80bc-369dc9932f96","Type":"ContainerStarted","Data":"3e26dc413cb34c2b7f3a57578bb3ed6c4a99db3a56a235d643c6b7769e296f42"}
Oct 30 19:01:13.205871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.205850    5232 generic.go:356] "Generic (PLEG): container finished" podID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerID="9a3d8f215c2dda4523205e7b1548a8b821331acb7e0641fcd48180bbe101c70f" exitCode=0
Oct 30 19:01:13.205966 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.205923    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" event={"ID":"7c5e8235-3b0b-4684-b9c8-7438399620ab","Type":"ContainerDied","Data":"9a3d8f215c2dda4523205e7b1548a8b821331acb7e0641fcd48180bbe101c70f"}
Oct 30 19:01:13.206013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.205971    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf" event={"ID":"7c5e8235-3b0b-4684-b9c8-7438399620ab","Type":"ContainerDied","Data":"54f79cd27fa11b5375bff0adbbde47cb68fe390598bbb29e363d094bd425ba44"}
Oct 30 19:01:13.206013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.205997    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54f79cd27fa11b5375bff0adbbde47cb68fe390598bbb29e363d094bd425ba44"
Oct 30 19:01:13.206077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.206015    5232 scope.go:122] "RemoveContainer" containerID="eaef2c5d17fbcc30ca48ef21bbdf448e012ffc7952b2512ea929725f2e2ea3e7"
Oct 30 19:01:13.221068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.221047    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 19:01:13.318634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318601    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-credential-socket\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.318842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318643    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-socket\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.318842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318684    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-certs\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.318842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318756    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-data\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.318842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318779    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-envoy\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.318842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318803    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-kube-api-access-x7xjs\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-kube-api-access-x7xjs\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.319111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318908    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-podinfo\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.319111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318931    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c5e8235-3b0b-4684-b9c8-7438399620ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c5e8235-3b0b-4684-b9c8-7438399620ab-istiod-ca-cert\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.319111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318948    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-token\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-token\") pod \"7c5e8235-3b0b-4684-b9c8-7438399620ab\" (UID: \"7c5e8235-3b0b-4684-b9c8-7438399620ab\") "
Oct 30 19:01:13.319111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.318950    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-credential-socket" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:13.319111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.319012    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-socket" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:13.319364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.319113    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-data" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:13.319427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.319366    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c5e8235-3b0b-4684-b9c8-7438399620ab-istiod-ca-cert" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:13.319571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.319545    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c5e8235-3b0b-4684-b9c8-7438399620ab-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.319571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.319572    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.320068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.319588    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.320068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.319602    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.320068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.319687    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-certs" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:13.321193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.321173    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-token" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:13.321376 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.321360    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-kube-api-access-x7xjs" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "kube-api-access-x7xjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:13.321678 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.321651    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-podinfo" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:01:13.322049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.322021    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-envoy" pod "7c5e8235-3b0b-4684-b9c8-7438399620ab" (UID: "7c5e8235-3b0b-4684-b9c8-7438399620ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:13.323367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.323329    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" podStartSLOduration=2.323317443 podStartE2EDuration="2.323317443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:13.322155722 +0000 UTC m=+865.906638720" watchObservedRunningTime="2026-10-30 19:01:13.323317443 +0000 UTC m=+865.907800440"
Oct 30 19:01:13.421137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.421105    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.421137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.421132    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.421137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.421143    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.421367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.421152    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c5e8235-3b0b-4684-b9c8-7438399620ab-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.421367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.421161    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7xjs\" (UniqueName: \"kubernetes.io/projected/7c5e8235-3b0b-4684-b9c8-7438399620ab-kube-api-access-x7xjs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:13.862631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.862594    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:13.867868 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:13.867841    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:14.213148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:14.213070    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"
Oct 30 19:01:14.213605 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:14.213586    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:14.214461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:14.214436    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:14.437968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:14.437930    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"]
Oct 30 19:01:14.442769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:14.442737    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e7b96f88-openshift-default-6df7d8d999-td7jf"]
Oct 30 19:01:15.538920 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:15.538883    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" path="/var/lib/kubelet/pods/7c5e8235-3b0b-4684-b9c8-7438399620ab/volumes"
Oct 30 19:01:16.186879 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:16.186845    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:01:16.817996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:16.817965    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"]
Oct 30 19:01:16.818441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:16.818183    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf" podUID="088bb2ad-1f04-4af8-825f-2114a17e85cf" containerName="maas-api" containerID="cri-o://4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051" gracePeriod=30
Oct 30 19:01:17.076157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.076103    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:01:17.158969 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.158940    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls\") pod \"088bb2ad-1f04-4af8-825f-2114a17e85cf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") "
Oct 30 19:01:17.159110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.158979    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/088bb2ad-1f04-4af8-825f-2114a17e85cf-kube-api-access-bmxjv\" (UniqueName: \"kubernetes.io/projected/088bb2ad-1f04-4af8-825f-2114a17e85cf-kube-api-access-bmxjv\") pod \"088bb2ad-1f04-4af8-825f-2114a17e85cf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") "
Oct 30 19:01:17.159169 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.159150    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-metrics-certs\") pod \"088bb2ad-1f04-4af8-825f-2114a17e85cf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") "
Oct 30 19:01:17.159249 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.159237    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/088bb2ad-1f04-4af8-825f-2114a17e85cf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/088bb2ad-1f04-4af8-825f-2114a17e85cf-openshift-service-ca\") pod \"088bb2ad-1f04-4af8-825f-2114a17e85cf\" (UID: \"088bb2ad-1f04-4af8-825f-2114a17e85cf\") "
Oct 30 19:01:17.159692 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.159653    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/088bb2ad-1f04-4af8-825f-2114a17e85cf-openshift-service-ca" pod "088bb2ad-1f04-4af8-825f-2114a17e85cf" (UID: "088bb2ad-1f04-4af8-825f-2114a17e85cf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:17.161139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.161112    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-metrics-certs" pod "088bb2ad-1f04-4af8-825f-2114a17e85cf" (UID: "088bb2ad-1f04-4af8-825f-2114a17e85cf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:17.161229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.161145    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls" pod "088bb2ad-1f04-4af8-825f-2114a17e85cf" (UID: "088bb2ad-1f04-4af8-825f-2114a17e85cf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:17.161229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.161163    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/088bb2ad-1f04-4af8-825f-2114a17e85cf-kube-api-access-bmxjv" pod "088bb2ad-1f04-4af8-825f-2114a17e85cf" (UID: "088bb2ad-1f04-4af8-825f-2114a17e85cf"). InnerVolumeSpecName "kube-api-access-bmxjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:17.233261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.233224    5232 generic.go:356] "Generic (PLEG): container finished" podID="088bb2ad-1f04-4af8-825f-2114a17e85cf" containerID="4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051" exitCode=0
Oct 30 19:01:17.233440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.233276    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"
Oct 30 19:01:17.233440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.233305    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf" event={"ID":"088bb2ad-1f04-4af8-825f-2114a17e85cf","Type":"ContainerDied","Data":"4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051"}
Oct 30 19:01:17.233440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.233360    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf" event={"ID":"088bb2ad-1f04-4af8-825f-2114a17e85cf","Type":"ContainerDied","Data":"fb70019bd82a47fd7536fc31e7b7a0c988e63295c81f84119e0ca4e2a4777ce1"}
Oct 30 19:01:17.233440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.233383    5232 scope.go:122] "RemoveContainer" containerID="4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051"
Oct 30 19:01:17.244153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.244134    5232 scope.go:122] "RemoveContainer" containerID="4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051"
Oct 30 19:01:17.244417 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:17.244395    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051\": container with ID starting with 4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051 not found: ID does not exist" containerID="4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051"
Oct 30 19:01:17.244463 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.244426    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051"} err="failed to get container status \"4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051\": rpc error: code = NotFound desc = could not find container \"4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051\": container with ID starting with 4701772cba7fad492491f792a866b6e628e9f4400393ff17dae05065f1b95051 not found: ID does not exist"
Oct 30 19:01:17.260549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.260521    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/088bb2ad-1f04-4af8-825f-2114a17e85cf-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.260549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.260547    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.260759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.260561    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmxjv\" (UniqueName: \"kubernetes.io/projected/088bb2ad-1f04-4af8-825f-2114a17e85cf-kube-api-access-bmxjv\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.260759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.260582    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/088bb2ad-1f04-4af8-825f-2114a17e85cf-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.392284 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.392251    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"]
Oct 30 19:01:17.396368 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.396343    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df563ff7-586dd7df89-njpkf"]
Oct 30 19:01:17.539050 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:17.539011    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="088bb2ad-1f04-4af8-825f-2114a17e85cf" path="/var/lib/kubelet/pods/088bb2ad-1f04-4af8-825f-2114a17e85cf/volumes"
Oct 30 19:01:18.801334 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801303    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"]
Oct 30 19:01:18.801737 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801728    5232 cpu_manager.go:414] "containerMap: removing container" podUID="088bb2ad-1f04-4af8-825f-2114a17e85cf" containerName="maas-api"
Oct 30 19:01:18.801779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801740    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="088bb2ad-1f04-4af8-825f-2114a17e85cf" containerName="maas-api"
Oct 30 19:01:18.801779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801759    5232 cpu_manager.go:414] "containerMap: removing container" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy"
Oct 30 19:01:18.801779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801764    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy"
Oct 30 19:01:18.801779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801772    5232 cpu_manager.go:414] "containerMap: removing container" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy"
Oct 30 19:01:18.801779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801777    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy"
Oct 30 19:01:18.801959 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801857    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="088bb2ad-1f04-4af8-825f-2114a17e85cf" containerName="maas-api"
Oct 30 19:01:18.801959 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801869    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy"
Oct 30 19:01:18.801959 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.801876    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c5e8235-3b0b-4684-b9c8-7438399620ab" containerName="istio-proxy"
Oct 30 19:01:18.806602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.806580    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:18.878005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.877707    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/827c3421-c6e8-404f-84b7-69cf1bd1088d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8\" (UID: \"827c3421-c6e8-404f-84b7-69cf1bd1088d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:18.878005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.877792    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxj9g\" (UniqueName: \"kubernetes.io/projected/827c3421-c6e8-404f-84b7-69cf1bd1088d-kube-api-access-fxj9g\") pod \"payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8\" (UID: \"827c3421-c6e8-404f-84b7-69cf1bd1088d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:18.905687 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.905652    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-9275c3\""
Oct 30 19:01:18.905876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.905746    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-9275c3-dockercfg-4tl6v\""
Oct 30 19:01:18.909397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.909372    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"]
Oct 30 19:01:18.978885 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.978846    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/827c3421-c6e8-404f-84b7-69cf1bd1088d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8\" (UID: \"827c3421-c6e8-404f-84b7-69cf1bd1088d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:18.979052 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.978927    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxj9g\" (UniqueName: \"kubernetes.io/projected/827c3421-c6e8-404f-84b7-69cf1bd1088d-kube-api-access-fxj9g\") pod \"payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8\" (UID: \"827c3421-c6e8-404f-84b7-69cf1bd1088d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:18.979538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:18.979520    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/827c3421-c6e8-404f-84b7-69cf1bd1088d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8\" (UID: \"827c3421-c6e8-404f-84b7-69cf1bd1088d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:19.038279 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:19.038246    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxj9g\" (UniqueName: \"kubernetes.io/projected/827c3421-c6e8-404f-84b7-69cf1bd1088d-kube-api-access-fxj9g\") pod \"payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8\" (UID: \"827c3421-c6e8-404f-84b7-69cf1bd1088d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:19.120102 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:19.120071    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:19.493202 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:19.493117    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"]
Oct 30 19:01:19.498643 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:01:19.498611    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827c3421_c6e8_404f_84b7_69cf1bd1088d.slice/crio-31ec280a77584367b48c650a2bd7c1162d5a15a8c7f6e1a312f7efa52ea4380b WatchSource:0}: Error finding container 31ec280a77584367b48c650a2bd7c1162d5a15a8c7f6e1a312f7efa52ea4380b: Status 404 returned error can't find the container with id 31ec280a77584367b48c650a2bd7c1162d5a15a8c7f6e1a312f7efa52ea4380b
Oct 30 19:01:20.252891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:20.252853    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8" event={"ID":"827c3421-c6e8-404f-84b7-69cf1bd1088d","Type":"ContainerStarted","Data":"9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115"}
Oct 30 19:01:20.253308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:20.253027    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:20.253308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:20.253047    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8" event={"ID":"827c3421-c6e8-404f-84b7-69cf1bd1088d","Type":"ContainerStarted","Data":"31ec280a77584367b48c650a2bd7c1162d5a15a8c7f6e1a312f7efa52ea4380b"}
Oct 30 19:01:20.366059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:20.366005    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8" podStartSLOduration=2.365992589 podStartE2EDuration="2.365992589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:20.365526808 +0000 UTC m=+872.950009806" watchObservedRunningTime="2026-10-30 19:01:20.365992589 +0000 UTC m=+872.950475585"
Oct 30 19:01:23.321657 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.321622    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"]
Oct 30 19:01:23.325923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.325899    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.379201 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.379168    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-9275c3-serving-cert\""
Oct 30 19:01:23.422771 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.422732    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv8ln\" (UniqueName: \"kubernetes.io/projected/198f0b29-1953-4a6b-88ee-638cd1f6e114-kube-api-access-jv8ln\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.422958 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.422784    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/198f0b29-1953-4a6b-88ee-638cd1f6e114-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.422958 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.422831    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-metrics-certs\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.423043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.422952    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.426151 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.426129    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7w45v\""
Oct 30 19:01:23.426273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.426188    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 19:01:23.428398 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.428380    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"]
Oct 30 19:01:23.524111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.524070    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/198f0b29-1953-4a6b-88ee-638cd1f6e114-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.524299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.524133    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-metrics-certs\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.524299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.524179    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.524299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.524272    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv8ln\" (UniqueName: \"kubernetes.io/projected/198f0b29-1953-4a6b-88ee-638cd1f6e114-kube-api-access-jv8ln\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.524472 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:23.524348    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-serving-cert: secret "maas-api-e2e-worker-w1-9275c3-serving-cert" not found
Oct 30 19:01:23.524472 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:23.524424    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls podName:198f0b29-1953-4a6b-88ee-638cd1f6e114 nodeName:}" failed. No retries permitted until 2026-10-30 19:01:24.024406676 +0000 UTC m=+876.608889670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls") pod "maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n" (UID: "198f0b29-1953-4a6b-88ee-638cd1f6e114") : secret "maas-api-e2e-worker-w1-9275c3-serving-cert" not found
Oct 30 19:01:23.524767 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.524747    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/198f0b29-1953-4a6b-88ee-638cd1f6e114-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.526489 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.526472    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-metrics-certs\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:23.584304 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:23.584226    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv8ln\" (UniqueName: \"kubernetes.io/projected/198f0b29-1953-4a6b-88ee-638cd1f6e114-kube-api-access-jv8ln\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:24.033071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:24.033028    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:24.035356 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:24.035326    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls\") pod \"maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:24.238942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:24.238901    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:24.613935 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:24.613897    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"]
Oct 30 19:01:24.622294 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:01:24.622266    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198f0b29_1953_4a6b_88ee_638cd1f6e114.slice/crio-6e6e23f80f3706aca0529b4bce1a70595f34cf943516daf897578132344a9a55 WatchSource:0}: Error finding container 6e6e23f80f3706aca0529b4bce1a70595f34cf943516daf897578132344a9a55: Status 404 returned error can't find the container with id 6e6e23f80f3706aca0529b4bce1a70595f34cf943516daf897578132344a9a55
Oct 30 19:01:25.281213 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:25.281160    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n" event={"ID":"198f0b29-1953-4a6b-88ee-638cd1f6e114","Type":"ContainerStarted","Data":"6e6e23f80f3706aca0529b4bce1a70595f34cf943516daf897578132344a9a55"}
Oct 30 19:01:26.288461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:26.288418    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n" event={"ID":"198f0b29-1953-4a6b-88ee-638cd1f6e114","Type":"ContainerStarted","Data":"3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077"}
Oct 30 19:01:26.288903 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:26.288534    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:26.405192 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:26.405142    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n" podStartSLOduration=2.57384852 podStartE2EDuration="3.40512919s" totalImagesPullingTime="831.28067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:26.403221415 +0000 UTC m=+878.987704409" watchObservedRunningTime="2026-10-30 19:01:26.40512919 +0000 UTC m=+878.989612194"
Oct 30 19:01:31.262653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:31.262623    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:01:32.300774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:32.300742    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:01:32.967681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:32.967647    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"]
Oct 30 19:01:32.971451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:32.971431    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:33.037588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.037560    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-1323a7\""
Oct 30 19:01:33.045653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.045630    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-1323a7-dockercfg-4j9xm\""
Oct 30 19:01:33.083905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.083877    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"]
Oct 30 19:01:33.085737 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.085711    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"]
Oct 30 19:01:33.089177 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.089155    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:33.122304 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.122273    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a2fe95-4f24-4210-bf32-7791b2de59b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm\" (UID: \"69a2fe95-4f24-4210-bf32-7791b2de59b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:33.122443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.122358    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpl8f\" (UniqueName: \"kubernetes.io/projected/69a2fe95-4f24-4210-bf32-7791b2de59b4-kube-api-access-xpl8f\") pod \"payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm\" (UID: \"69a2fe95-4f24-4210-bf32-7791b2de59b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:33.145732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.145694    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"]
Oct 30 19:01:33.223234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.223150    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58c132fb-563a-4f6f-ae7c-2a6074100f8b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564\" (UID: \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:33.223234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.223202    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpl8f\" (UniqueName: \"kubernetes.io/projected/69a2fe95-4f24-4210-bf32-7791b2de59b4-kube-api-access-xpl8f\") pod \"payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm\" (UID: \"69a2fe95-4f24-4210-bf32-7791b2de59b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:33.223455 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.223282    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a2fe95-4f24-4210-bf32-7791b2de59b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm\" (UID: \"69a2fe95-4f24-4210-bf32-7791b2de59b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:33.223455 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.223300    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhw65\" (UniqueName: \"kubernetes.io/projected/58c132fb-563a-4f6f-ae7c-2a6074100f8b-kube-api-access-hhw65\") pod \"payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564\" (UID: \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:33.224073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.224047    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a2fe95-4f24-4210-bf32-7791b2de59b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm\" (UID: \"69a2fe95-4f24-4210-bf32-7791b2de59b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:33.283785 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.283745    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpl8f\" (UniqueName: \"kubernetes.io/projected/69a2fe95-4f24-4210-bf32-7791b2de59b4-kube-api-access-xpl8f\") pod \"payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm\" (UID: \"69a2fe95-4f24-4210-bf32-7791b2de59b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:33.283987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.283949    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:33.327828 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.327198    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhw65\" (UniqueName: \"kubernetes.io/projected/58c132fb-563a-4f6f-ae7c-2a6074100f8b-kube-api-access-hhw65\") pod \"payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564\" (UID: \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:33.327828 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.327290    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58c132fb-563a-4f6f-ae7c-2a6074100f8b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564\" (UID: \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:33.333703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.329542    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58c132fb-563a-4f6f-ae7c-2a6074100f8b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564\" (UID: \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:33.387335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.387297    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhw65\" (UniqueName: \"kubernetes.io/projected/58c132fb-563a-4f6f-ae7c-2a6074100f8b-kube-api-access-hhw65\") pod \"payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564\" (UID: \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:33.402583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.402558    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:33.657887 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.657856    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"]
Oct 30 19:01:33.663005 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:01:33.662972    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a2fe95_4f24_4210_bf32_7791b2de59b4.slice/crio-7939ce77a7194a03a35627cd9f7e12212f224d01b421b58493d8ef627d45fc9e WatchSource:0}: Error finding container 7939ce77a7194a03a35627cd9f7e12212f224d01b421b58493d8ef627d45fc9e: Status 404 returned error can't find the container with id 7939ce77a7194a03a35627cd9f7e12212f224d01b421b58493d8ef627d45fc9e
Oct 30 19:01:33.774056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:33.774016    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"]
Oct 30 19:01:33.778748 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:01:33.778723    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c132fb_563a_4f6f_ae7c_2a6074100f8b.slice/crio-8667a91e34124f0758138724f2bf2e5c8ef7ebe629abfd7272591efd212177c0 WatchSource:0}: Error finding container 8667a91e34124f0758138724f2bf2e5c8ef7ebe629abfd7272591efd212177c0: Status 404 returned error can't find the container with id 8667a91e34124f0758138724f2bf2e5c8ef7ebe629abfd7272591efd212177c0
Oct 30 19:01:34.346562 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:34.346529    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564" event={"ID":"58c132fb-563a-4f6f-ae7c-2a6074100f8b","Type":"ContainerStarted","Data":"1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e"}
Oct 30 19:01:34.347046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:34.346580    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564" event={"ID":"58c132fb-563a-4f6f-ae7c-2a6074100f8b","Type":"ContainerStarted","Data":"8667a91e34124f0758138724f2bf2e5c8ef7ebe629abfd7272591efd212177c0"}
Oct 30 19:01:34.347046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:34.346732    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:34.348386 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:34.348339    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm" event={"ID":"69a2fe95-4f24-4210-bf32-7791b2de59b4","Type":"ContainerStarted","Data":"0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5"}
Oct 30 19:01:34.348514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:34.348409    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm" event={"ID":"69a2fe95-4f24-4210-bf32-7791b2de59b4","Type":"ContainerStarted","Data":"7939ce77a7194a03a35627cd9f7e12212f224d01b421b58493d8ef627d45fc9e"}
Oct 30 19:01:34.348514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:34.348447    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:34.461935 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:34.461884    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564" podStartSLOduration=1.461871039 podStartE2EDuration="1.461871039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:34.460775081 +0000 UTC m=+887.045258082" watchObservedRunningTime="2026-10-30 19:01:34.461871039 +0000 UTC m=+887.046354059"
Oct 30 19:01:34.576720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:34.576668    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm" podStartSLOduration=2.576646775 podStartE2EDuration="2.576646775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:34.574467133 +0000 UTC m=+887.158950122" watchObservedRunningTime="2026-10-30 19:01:34.576646775 +0000 UTC m=+887.161129772"
Oct 30 19:01:40.404174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.404135    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"]
Oct 30 19:01:40.408620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.408594    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:40.461239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.461189    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-330025-dockercfg-g42nl\""
Oct 30 19:01:40.496780 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.496741    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87\" (UID: \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:40.497012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.496803    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq49d\" (UniqueName: \"kubernetes.io/projected/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-kube-api-access-bq49d\") pod \"payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87\" (UID: \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:40.508704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.508681    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-330025\""
Oct 30 19:01:40.511263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.511237    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"]
Oct 30 19:01:40.597283 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.597242    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87\" (UID: \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:40.597283 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.597287    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq49d\" (UniqueName: \"kubernetes.io/projected/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-kube-api-access-bq49d\") pod \"payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87\" (UID: \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:40.597963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.597942    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87\" (UID: \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:40.664789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.664713    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq49d\" (UniqueName: \"kubernetes.io/projected/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-kube-api-access-bq49d\") pod \"payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87\" (UID: \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:40.722092 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:40.722057    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:41.123865 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.123801    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"]
Oct 30 19:01:41.128581 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:01:41.128551    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e7fe22_59ed_43d3_8fc8_36b88aa829a3.slice/crio-f318b41445ad37e12234d8e34fb9877d9101f3ec8177ebdc350133d13451b573 WatchSource:0}: Error finding container f318b41445ad37e12234d8e34fb9877d9101f3ec8177ebdc350133d13451b573: Status 404 returned error can't find the container with id f318b41445ad37e12234d8e34fb9877d9101f3ec8177ebdc350133d13451b573
Oct 30 19:01:41.394301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.393778    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87" event={"ID":"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3","Type":"ContainerStarted","Data":"1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081"}
Oct 30 19:01:41.394301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.393980    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:41.394301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.394018    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87" event={"ID":"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3","Type":"ContainerStarted","Data":"f318b41445ad37e12234d8e34fb9877d9101f3ec8177ebdc350133d13451b573"}
Oct 30 19:01:41.522779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.522724    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87" podStartSLOduration=1.522708553 podStartE2EDuration="1.522708553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:41.521486144 +0000 UTC m=+894.105969144" watchObservedRunningTime="2026-10-30 19:01:41.522708553 +0000 UTC m=+894.107191550"
Oct 30 19:01:41.667190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.667092    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"]
Oct 30 19:01:41.670958 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.670932    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.711576 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.711529    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.711863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.711594    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.711863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.711678    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e69a2eb3-19ad-417f-854e-49b0c11167a6-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.711863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.711780    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmr49\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-kube-api-access-gmr49\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.712043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.711928    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.712043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.711970    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.712124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.712081    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.712162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.712123    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.712195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.712164    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.722855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.722830    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2301cb-openshift-default-dockercfg-r29tl\""
Oct 30 19:01:41.732449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.732421    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"]
Oct 30 19:01:41.812666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812624    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.812886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812684    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.812886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812712    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.812886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812737    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.812886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812760    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e69a2eb3-19ad-417f-854e-49b0c11167a6-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.812886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812782    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmr49\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-kube-api-access-gmr49\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.812886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812883    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.813224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812912    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.813224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.812971    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.813224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.813101    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.813481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.813413    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.813481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.813460    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e69a2eb3-19ad-417f-854e-49b0c11167a6-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.813723 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.813682    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.813895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.813761    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.815947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.815922    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.817866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.817846    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.871611 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.871572    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.871842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.871691    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmr49\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-kube-api-access-gmr49\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:41.985200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:41.985117    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:42.463124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:42.463075    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"]
Oct 30 19:01:42.467169 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:01:42.467137    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode69a2eb3_19ad_417f_854e_49b0c11167a6.slice/crio-453ffe061c3b518aacde16f3226a385be8357a21ad7de1f68f4eac70bb2a75b7 WatchSource:0}: Error finding container 453ffe061c3b518aacde16f3226a385be8357a21ad7de1f68f4eac70bb2a75b7: Status 404 returned error can't find the container with id 453ffe061c3b518aacde16f3226a385be8357a21ad7de1f68f4eac70bb2a75b7
Oct 30 19:01:42.469402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:42.469368    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:42.469494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:42.469433    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:42.469494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:42.469465    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:43.408617 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:43.408583    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" event={"ID":"e69a2eb3-19ad-417f-854e-49b0c11167a6","Type":"ContainerStarted","Data":"aa3cd0ffb061e801d5978981a2bf46a8b1003951550099b12bd750fcedbd3401"}
Oct 30 19:01:43.409062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:43.408636    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" event={"ID":"e69a2eb3-19ad-417f-854e-49b0c11167a6","Type":"ContainerStarted","Data":"453ffe061c3b518aacde16f3226a385be8357a21ad7de1f68f4eac70bb2a75b7"}
Oct 30 19:01:43.625958 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:43.625907    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podStartSLOduration=2.625892426 podStartE2EDuration="2.625892426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:43.577142601 +0000 UTC m=+896.161625611" watchObservedRunningTime="2026-10-30 19:01:43.625892426 +0000 UTC m=+896.210375426"
Oct 30 19:01:43.985364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:43.985327    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:43.990275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:43.990251    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:44.414274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:44.414239    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:44.415305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:44.415287    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:01:45.358292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:45.358262    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 19:01:45.358477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:45.358340    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 19:01:46.187483 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:01:46.187444    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:01:47.616866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:47.616833    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:01:47.624280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:47.624257    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:01:47.624877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:47.624853    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:01:47.629885 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:47.629862    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:01:48.442327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:48.442290    5232 generic.go:356] "Generic (PLEG): container finished" podID="5745ae03-a721-4681-80bc-369dc9932f96" containerID="9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19" exitCode=0
Oct 30 19:01:48.442523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:48.442367    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" event={"ID":"5745ae03-a721-4681-80bc-369dc9932f96","Type":"ContainerDied","Data":"9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19"}
Oct 30 19:01:48.442791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:48.442774    5232 scope.go:122] "RemoveContainer" containerID="9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19"
Oct 30 19:01:48.444285 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:48.444250    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:48.444402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:48.444321    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:48.444402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:48.444355    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:49.469376 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:49.469340    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" event={"ID":"5745ae03-a721-4681-80bc-369dc9932f96","Type":"ContainerStarted","Data":"c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec"}
Oct 30 19:01:49.862291 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:49.862250    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:49.862477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:49.862341    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:01:49.862477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:49.862425    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 19:01:50.862203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:50.862168    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:01:50.862648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:50.862228    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 19:01:51.862253 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:51.862212    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:01:51.862660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:51.862297    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 19:01:52.404804 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:52.404774    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 19:01:52.863622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:52.863584    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:01:52.864288 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:52.864235    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 19:01:53.862283 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:53.862242    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:01:53.862467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:53.862322    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 19:01:54.866297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:54.866266    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:54.866666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:54.866535    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:01:54.867379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:01:54.867363    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:02:02.802456 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:02.802366    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"]
Oct 30 19:02:02.806548 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:02.806524    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:02.859837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:02.859227    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-2301cb\""
Oct 30 19:02:02.906899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:02.906866    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-2301cb-dockercfg-tvjxz\""
Oct 30 19:02:02.909221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:02.909195    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"]
Oct 30 19:02:02.977798 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:02.977758    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56f9b6fd-b4e9-495c-bafe-71522b3d3469-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2\" (UID: \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:02.978010 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:02.977837    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7xbm\" (UniqueName: \"kubernetes.io/projected/56f9b6fd-b4e9-495c-bafe-71522b3d3469-kube-api-access-g7xbm\") pod \"payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2\" (UID: \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:03.078731 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:03.078624    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7xbm\" (UniqueName: \"kubernetes.io/projected/56f9b6fd-b4e9-495c-bafe-71522b3d3469-kube-api-access-g7xbm\") pod \"payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2\" (UID: \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:03.078954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:03.078791    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56f9b6fd-b4e9-495c-bafe-71522b3d3469-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2\" (UID: \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:03.079516 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:03.079494    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56f9b6fd-b4e9-495c-bafe-71522b3d3469-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2\" (UID: \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:03.157404 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:03.157355    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7xbm\" (UniqueName: \"kubernetes.io/projected/56f9b6fd-b4e9-495c-bafe-71522b3d3469-kube-api-access-g7xbm\") pod \"payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2\" (UID: \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:03.421224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:03.421191    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:03.822200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:03.822116    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"]
Oct 30 19:02:03.827273 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:02:03.827229    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56f9b6fd_b4e9_495c_bafe_71522b3d3469.slice/crio-cf410e48daf73c729e6ae0b0a0bed21dff2d62e91b71208ad2b015fa4905f771 WatchSource:0}: Error finding container cf410e48daf73c729e6ae0b0a0bed21dff2d62e91b71208ad2b015fa4905f771: Status 404 returned error can't find the container with id cf410e48daf73c729e6ae0b0a0bed21dff2d62e91b71208ad2b015fa4905f771
Oct 30 19:02:04.578656 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:04.578613    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2" event={"ID":"56f9b6fd-b4e9-495c-bafe-71522b3d3469","Type":"ContainerStarted","Data":"3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd"}
Oct 30 19:02:04.578874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:04.578675    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2" event={"ID":"56f9b6fd-b4e9-495c-bafe-71522b3d3469","Type":"ContainerStarted","Data":"cf410e48daf73c729e6ae0b0a0bed21dff2d62e91b71208ad2b015fa4905f771"}
Oct 30 19:02:04.578874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:04.578717    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:04.693760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:04.693704    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2" podStartSLOduration=2.693690541 podStartE2EDuration="2.693690541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:04.692115522 +0000 UTC m=+917.276598521" watchObservedRunningTime="2026-10-30 19:02:04.693690541 +0000 UTC m=+917.278173537"
Oct 30 19:02:10.612679 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:10.612644    5232 generic.go:356] "Generic (PLEG): container finished" podID="e20ab810-41dd-465a-8883-08f583b528ee" containerID="5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d" exitCode=0
Oct 30 19:02:10.613100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:10.612717    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" event={"ID":"e20ab810-41dd-465a-8883-08f583b528ee","Type":"ContainerDied","Data":"5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d"}
Oct 30 19:02:10.613201 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:10.613185    5232 scope.go:122] "RemoveContainer" containerID="5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d"
Oct 30 19:02:10.614569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:10.614542    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:10.614644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:10.614613    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:10.614680 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:10.614647    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:11.620034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:11.619991    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" event={"ID":"e20ab810-41dd-465a-8883-08f583b528ee","Type":"ContainerStarted","Data":"9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5"}
Oct 30 19:02:11.627220 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:11.627197    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:02:11.628450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:11.628425    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:02:11.628574 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:11.628476    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 19:02:12.197492 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:12.197453    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:02:12.627000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:12.626968    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:02:12.627432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:12.627018    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 19:02:13.627392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:13.627359    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:02:13.627835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:13.627416    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 19:02:14.627006 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:14.626971    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:02:14.627220 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:14.627025    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 19:02:15.588012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:15.587983    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:02:15.627442 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:15.627409    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:02:15.627611 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:15.627475    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 19:02:16.187782 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:02:16.187740    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:02:16.627315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:16.627270    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:02:16.627799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:16.627346    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 19:02:17.631919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:17.631888    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:02:17.632992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:17.632971    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:02:40.830434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:40.830321    5232 generic.go:356] "Generic (PLEG): container finished" podID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerID="aa3cd0ffb061e801d5978981a2bf46a8b1003951550099b12bd750fcedbd3401" exitCode=0
Oct 30 19:02:40.830434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:40.830393    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" event={"ID":"e69a2eb3-19ad-417f-854e-49b0c11167a6","Type":"ContainerDied","Data":"aa3cd0ffb061e801d5978981a2bf46a8b1003951550099b12bd750fcedbd3401"}
Oct 30 19:02:40.831034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:40.830879    5232 scope.go:122] "RemoveContainer" containerID="aa3cd0ffb061e801d5978981a2bf46a8b1003951550099b12bd750fcedbd3401"
Oct 30 19:02:40.832540 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:40.832473    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:40.832733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:40.832641    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:40.832801 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:40.832721    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:41.863739 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:41.863706    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" event={"ID":"e69a2eb3-19ad-417f-854e-49b0c11167a6","Type":"ContainerStarted","Data":"3daf776f6b0a58a65e458e9640ad02bef3d91ab785be57b052f6872c4ae30eee"}
Oct 30 19:02:41.985527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:41.985490    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:02:41.986963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:41.986934    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:02:41.987081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:41.986984    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:02:42.986301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:42.986264    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:02:42.986709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:42.986316    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:02:43.986567 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:43.986530    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:02:43.987106 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:43.986604    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:02:44.415185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:44.415148    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:02:44.985753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:44.985713    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:02:44.985953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:44.985778    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:02:45.985733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:45.985697    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:02:45.986220 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:45.985780    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:02:46.189216 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:02:46.189180    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:02:46.985783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:46.985745    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:02:46.986217 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:46.985860    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:02:47.989631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:47.989602    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:02:47.990654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:02:47.990636    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:03:16.186696 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:03:16.186651    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:03:30.260091 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.260013    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"]
Oct 30 19:03:30.263256 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.263231    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.361280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.361245    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-22fb18\"/\"kube-root-ca.crt\""
Oct 30 19:03:30.361457 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.361291    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-22fb18\"/\"default-dockercfg-tj7q4\""
Oct 30 19:03:30.361457 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.361295    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-22fb18\"/\"premium-w3-22fb18-kserve-self-signed-certs\""
Oct 30 19:03:30.361457 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.361275    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-22fb18\"/\"openshift-service-ca.crt\""
Oct 30 19:03:30.363707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.363690    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"]
Oct 30 19:03:30.366900 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.366880    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ae7696-22ae-4fee-b71c-5741583f4f80-tls-certs\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.367004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.366950    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-dshm\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.367068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.366997    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qcsx\" (UniqueName: \"kubernetes.io/projected/47ae7696-22ae-4fee-b71c-5741583f4f80-kube-api-access-4qcsx\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.367129 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.367109    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-tmp-dir\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.367188 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.367163    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-home\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.367296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.367281    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-model-cache\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.468481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.468445    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-dshm\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.468481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.468482    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qcsx\" (UniqueName: \"kubernetes.io/projected/47ae7696-22ae-4fee-b71c-5741583f4f80-kube-api-access-4qcsx\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.468730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.468501    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-tmp-dir\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.468730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.468630    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-home\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.468730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.468728    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-model-cache\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.468934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.468799    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ae7696-22ae-4fee-b71c-5741583f4f80-tls-certs\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.468990 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.468929    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-tmp-dir\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.469031 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.469012    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-home\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.469090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.469070    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-model-cache\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.470740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.470716    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-dshm\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.471095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.471078    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ae7696-22ae-4fee-b71c-5741583f4f80-tls-certs\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.527136 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.527068    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qcsx\" (UniqueName: \"kubernetes.io/projected/47ae7696-22ae-4fee-b71c-5741583f4f80-kube-api-access-4qcsx\") pod \"premium-w3-22fb18-kserve-5785fbdd8-x2rbr\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") " pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.574255 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.574221    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:30.968609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:30.968564    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"]
Oct 30 19:03:30.973750 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:03:30.973720    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47ae7696_22ae_4fee_b71c_5741583f4f80.slice/crio-9dd30ec83d0d087120009eb84626c9815b66db18f7b44cbb834c262c808ca82e WatchSource:0}: Error finding container 9dd30ec83d0d087120009eb84626c9815b66db18f7b44cbb834c262c808ca82e: Status 404 returned error can't find the container with id 9dd30ec83d0d087120009eb84626c9815b66db18f7b44cbb834c262c808ca82e
Oct 30 19:03:31.148945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:31.148909    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" event={"ID":"47ae7696-22ae-4fee-b71c-5741583f4f80","Type":"ContainerStarted","Data":"a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f"}
Oct 30 19:03:31.148945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:31.148952    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" event={"ID":"47ae7696-22ae-4fee-b71c-5741583f4f80","Type":"ContainerStarted","Data":"9dd30ec83d0d087120009eb84626c9815b66db18f7b44cbb834c262c808ca82e"}
Oct 30 19:03:31.275749 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:31.275647    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podStartSLOduration=1.27562934 podStartE2EDuration="1.27562934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:31.27384495 +0000 UTC m=+1003.858327941" watchObservedRunningTime="2026-10-30 19:03:31.27562934 +0000 UTC m=+1003.860112334"
Oct 30 19:03:31.575314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:31.575236    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:40.575079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:40.575037    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:40.587483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:40.587452    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:41.218142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:41.218114    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:03:46.186842 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:03:46.186791    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:03:59.143243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:59.143205    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"]
Oct 30 19:03:59.143776 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:03:59.143465    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy" containerID="cri-o://1880ba3bafbb0c15ac3a249f4912afd07bb7436d8e3b44ce8452131fb88814dd" gracePeriod=30
Oct 30 19:04:04.343565 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.343514    5232 generic.go:356] "Generic (PLEG): container finished" podID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerID="1880ba3bafbb0c15ac3a249f4912afd07bb7436d8e3b44ce8452131fb88814dd" exitCode=0
Oct 30 19:04:04.344001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.343730    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" event={"ID":"2716fd62-07ab-4d55-a7a4-41610ef78a12","Type":"ContainerDied","Data":"1880ba3bafbb0c15ac3a249f4912afd07bb7436d8e3b44ce8452131fb88814dd"}
Oct 30 19:04:04.390598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.390570    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 19:04:04.494432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494323    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-token\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-token\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.494432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494379    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-envoy\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.494432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494422    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-socket\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.494726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494459    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-credential-socket\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.494726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494495    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2716fd62-07ab-4d55-a7a4-41610ef78a12-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2716fd62-07ab-4d55-a7a4-41610ef78a12-istiod-ca-cert\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.494726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494533    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-data\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.494726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494586    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-kube-api-access-nthbc\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-kube-api-access-nthbc\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.494726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494670    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-certs\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494729    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-podinfo\") pod \"2716fd62-07ab-4d55-a7a4-41610ef78a12\" (UID: \"2716fd62-07ab-4d55-a7a4-41610ef78a12\") "
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.494995    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-credential-socket" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.495010    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2716fd62-07ab-4d55-a7a4-41610ef78a12-istiod-ca-cert" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.495101    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-certs" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.495194    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.495217    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.495233    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2716fd62-07ab-4d55-a7a4-41610ef78a12-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.495285    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-socket" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:04.495604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.495315    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-data" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:04.496994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.496971    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-podinfo" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:04:04.497156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.497132    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-envoy" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:04.497320 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.497299    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-kube-api-access-nthbc" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "kube-api-access-nthbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:04.497400 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.497331    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-token" pod "2716fd62-07ab-4d55-a7a4-41610ef78a12" (UID: "2716fd62-07ab-4d55-a7a4-41610ef78a12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:04.595786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.595743    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:04.595786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.595783    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:04.595786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.595794    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:04.596034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.595803    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:04.596034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.595829    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nthbc\" (UniqueName: \"kubernetes.io/projected/2716fd62-07ab-4d55-a7a4-41610ef78a12-kube-api-access-nthbc\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:04.596034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:04.595840    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2716fd62-07ab-4d55-a7a4-41610ef78a12-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:05.351138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:05.351101    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj" event={"ID":"2716fd62-07ab-4d55-a7a4-41610ef78a12","Type":"ContainerDied","Data":"1f88d976e0da9c21ffa9c5febd56dc1199930f537879bbeee18a27770f38f83b"}
Oct 30 19:04:05.351601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:05.351170    5232 scope.go:122] "RemoveContainer" containerID="1880ba3bafbb0c15ac3a249f4912afd07bb7436d8e3b44ce8452131fb88814dd"
Oct 30 19:04:05.351601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:05.351122    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"
Oct 30 19:04:05.564425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:05.563951    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"]
Oct 30 19:04:05.571678 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:05.571648    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-t42hj"]
Oct 30 19:04:07.538893 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:07.538865    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" path="/var/lib/kubelet/pods/2716fd62-07ab-4d55-a7a4-41610ef78a12/volumes"
Oct 30 19:04:10.005883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.005843    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"]
Oct 30 19:04:10.006627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.006597    5232 cpu_manager.go:414] "containerMap: removing container" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy"
Oct 30 19:04:10.006743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.006633    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy"
Oct 30 19:04:10.006799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.006776    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="2716fd62-07ab-4d55-a7a4-41610ef78a12" containerName="istio-proxy"
Oct 30 19:04:10.010535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.010504    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.055535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.055498    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tmp-dir\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.055721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.055562    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tls-certs\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.055721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.055588    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-home\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.055721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.055612    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-model-cache\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.055721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.055650    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sh8n\" (UniqueName: \"kubernetes.io/projected/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-kube-api-access-6sh8n\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.055721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.055679    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-dshm\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.061101 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.061075    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-330025\"/\"openshift-service-ca.crt\""
Oct 30 19:04:10.108384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.108356    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-330025\"/\"kube-root-ca.crt\""
Oct 30 19:04:10.108675 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.108650    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-330025\"/\"premium-w2-330025-kserve-self-signed-certs\""
Oct 30 19:04:10.108775 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.108761    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-330025\"/\"default-dockercfg-77fzp\""
Oct 30 19:04:10.111483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.111456    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"]
Oct 30 19:04:10.156363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156326    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tmp-dir\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.156584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156377    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tls-certs\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.156584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156404    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-home\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.156584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156442    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-model-cache\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.156584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156480    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sh8n\" (UniqueName: \"kubernetes.io/projected/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-kube-api-access-6sh8n\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.156584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156510    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-dshm\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.156907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156827    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tmp-dir\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.156907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156871    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-home\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.157003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.156946    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-model-cache\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.158904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.158878    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tls-certs\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.159083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.159067    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-dshm\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.215745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.215713    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sh8n\" (UniqueName: \"kubernetes.io/projected/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-kube-api-access-6sh8n\") pod \"premium-w2-330025-kserve-79956d98fc-lgwbm\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") " pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.327232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.327149    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:10.743150 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:10.743110    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"]
Oct 30 19:04:11.394461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:11.394420    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" event={"ID":"bbabad87-3446-4c59-94a0-ef9bd2cf31c0","Type":"ContainerStarted","Data":"2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c"}
Oct 30 19:04:11.394839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:11.394481    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" event={"ID":"bbabad87-3446-4c59-94a0-ef9bd2cf31c0","Type":"ContainerStarted","Data":"50ae493dbc39caa34319da88f0d95d6d3bc12531bd0cd4b3951e2181dd1d88d4"}
Oct 30 19:04:11.508349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:11.508295    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podStartSLOduration=2.508281748 podStartE2EDuration="2.508281748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:11.508027551 +0000 UTC m=+1044.092510548" watchObservedRunningTime="2026-10-30 19:04:11.508281748 +0000 UTC m=+1044.092764745"
Oct 30 19:04:12.328018 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:12.327974    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:16.187373 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:04:16.187336    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:04:20.328410 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:20.328384    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:20.341402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:20.341374    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:20.458478 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:20.458447    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:04:36.268294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.268258    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"]
Oct 30 19:04:36.272704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.272660    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.294062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.294032    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tls-certs\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.294197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.294080    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tmp-dir\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.294197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.294146    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-dshm\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.294273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.294216    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-home\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.294273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.294248    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqhdt\" (UniqueName: \"kubernetes.io/projected/f9b68dad-e4e3-4892-924a-2cb5c605fa18-kube-api-access-pqhdt\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.294375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.294317    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-model-cache\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.349945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.349906    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"kube-root-ca.crt\""
Oct 30 19:04:36.373916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.373734    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"openshift-service-ca.crt\""
Oct 30 19:04:36.373916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.373879    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"default-dockercfg-6c9n4\""
Oct 30 19:04:36.374295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.374271    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"premium-w0-2301cb-kserve-self-signed-certs\""
Oct 30 19:04:36.383447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.383418    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"]
Oct 30 19:04:36.395658 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.395623    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tls-certs\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.395839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.395697    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tmp-dir\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.395839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.395732    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-dshm\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.395970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.395858    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-home\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.395970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.395883    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqhdt\" (UniqueName: \"kubernetes.io/projected/f9b68dad-e4e3-4892-924a-2cb5c605fa18-kube-api-access-pqhdt\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.395970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.395944    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-model-cache\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.396386 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.396360    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-model-cache\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.396907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.396556    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tmp-dir\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.397053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.396807    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-home\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.398076 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.398035    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tls-certs\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.399046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.399024    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-dshm\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.459315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.459285    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqhdt\" (UniqueName: \"kubernetes.io/projected/f9b68dad-e4e3-4892-924a-2cb5c605fa18-kube-api-access-pqhdt\") pod \"premium-w0-2301cb-kserve-676d6d78d5-kj64p\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") " pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:36.586954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:36.586929    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:37.003034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:37.002949    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"]
Oct 30 19:04:37.006399 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:04:37.006370    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9b68dad_e4e3_4892_924a_2cb5c605fa18.slice/crio-e6af518369878699cd3894c0ca426cd2c46143554b5ae74d90f7e220f19a91d1 WatchSource:0}: Error finding container e6af518369878699cd3894c0ca426cd2c46143554b5ae74d90f7e220f19a91d1: Status 404 returned error can't find the container with id e6af518369878699cd3894c0ca426cd2c46143554b5ae74d90f7e220f19a91d1
Oct 30 19:04:37.554835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:37.554750    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" event={"ID":"f9b68dad-e4e3-4892-924a-2cb5c605fa18","Type":"ContainerStarted","Data":"de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae"}
Oct 30 19:04:37.554835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:37.554825    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" event={"ID":"f9b68dad-e4e3-4892-924a-2cb5c605fa18","Type":"ContainerStarted","Data":"e6af518369878699cd3894c0ca426cd2c46143554b5ae74d90f7e220f19a91d1"}
Oct 30 19:04:37.587540 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:37.587507    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:46.188060 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:04:46.188019    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:04:46.587917 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:46.587879    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:46.606173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:46.606142    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:46.619521 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:46.619495    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:04:46.723861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:04:46.723763    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podStartSLOduration=10.723748843 podStartE2EDuration="10.723748843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:37.681083532 +0000 UTC m=+1070.265566529" watchObservedRunningTime="2026-10-30 19:04:46.723748843 +0000 UTC m=+1079.308231849"
Oct 30 19:05:16.186712 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:05:16.186675    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:05:40.419803 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.419766    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"]
Oct 30 19:05:40.423679 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.423655    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.474134 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.474105    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-7378d4fb8020\""
Oct 30 19:05:40.474134 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.474121    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-c159be30ecbc\""
Oct 30 19:05:40.521651 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.521618    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 19:05:40.521651 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.521649    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 19:05:40.521651 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.521649    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 19:05:40.521923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.521689    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 19:05:40.521923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.521728    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-bvv24\""
Oct 30 19:05:40.524474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.524455    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"]
Oct 30 19:05:40.612793 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.612762    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-routing-overlay-volume\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.612979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.612941    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.612979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.612975    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-plugins-config-volume\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.613082 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.612996    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkc6k\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-kube-api-access-wkc6k\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.714467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.714368    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.714467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.714417    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-plugins-config-volume\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.714467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.714450    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkc6k\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-kube-api-access-wkc6k\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.714771 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.714618    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-routing-overlay-volume\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.715236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.715216    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-routing-overlay-volume\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.715294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.715276    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-plugins-config-volume\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.717055 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.717022    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.773848 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.773796    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkc6k\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-kube-api-access-wkc6k\") pod \"payload-processing-external-model-5469ccd947-b2lgj\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") " pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:40.967038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.966925    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"]
Oct 30 19:05:40.971022 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:40.970999    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.021234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.021201    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-1323a7-openshift-default-dockercfg-2p7xv\""
Oct 30 19:05:41.036532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.036505    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:41.075519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.075482    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"]
Oct 30 19:05:41.119608 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119576    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.119766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119632    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-494gt\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-kube-api-access-494gt\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.119766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119674    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.119766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119706    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66491f90-bac7-4ddf-8452-6471424bc4af-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.119766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119738    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66491f90-bac7-4ddf-8452-6471424bc4af-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.119766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119756    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.120067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119781    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.120067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119830    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.120067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.119904    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.221745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.221663    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66491f90-bac7-4ddf-8452-6471424bc4af-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.221745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.221719    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66491f90-bac7-4ddf-8452-6471424bc4af-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.221973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.221883    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.221973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.221925    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.221973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.221969    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.221991    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.222036    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.222091    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-494gt\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-kube-api-access-494gt\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222279 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.222158    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222336 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.222321    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222390 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.222357    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222472 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.222443    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66491f90-bac7-4ddf-8452-6471424bc4af-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.222509    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.222590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.222573    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.224196 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.224173    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.224355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.224279    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66491f90-bac7-4ddf-8452-6471424bc4af-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.282068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.282031    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.282221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.282138    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-494gt\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-kube-api-access-494gt\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.284892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.284874    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:41.415505 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.415474    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"]
Oct 30 19:05:41.420348 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:05:41.420320    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d16144_6aaa_4029_9428_bfb1fb676dbb.slice/crio-8f22b9d01c04c74b2139408fcd5920bf71db3bbaa810237bb048addc0961df26 WatchSource:0}: Error finding container 8f22b9d01c04c74b2139408fcd5920bf71db3bbaa810237bb048addc0961df26: Status 404 returned error can't find the container with id 8f22b9d01c04c74b2139408fcd5920bf71db3bbaa810237bb048addc0961df26
Oct 30 19:05:41.660940 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.660903    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"]
Oct 30 19:05:41.665937 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:05:41.665905    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66491f90_bac7_4ddf_8452_6471424bc4af.slice/crio-b334be42413cd4ec7a9dcaf5d4b204cbab458a31dc2fd060969bce7a9caa1f40 WatchSource:0}: Error finding container b334be42413cd4ec7a9dcaf5d4b204cbab458a31dc2fd060969bce7a9caa1f40: Status 404 returned error can't find the container with id b334be42413cd4ec7a9dcaf5d4b204cbab458a31dc2fd060969bce7a9caa1f40
Oct 30 19:05:41.668312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.668279    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:05:41.668397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.668344    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:05:41.668397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.668376    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:05:41.939849 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.939772    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" event={"ID":"66491f90-bac7-4ddf-8452-6471424bc4af","Type":"ContainerStarted","Data":"af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb"}
Oct 30 19:05:41.940064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.939905    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" event={"ID":"66491f90-bac7-4ddf-8452-6471424bc4af","Type":"ContainerStarted","Data":"b334be42413cd4ec7a9dcaf5d4b204cbab458a31dc2fd060969bce7a9caa1f40"}
Oct 30 19:05:41.941467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.941436    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj" event={"ID":"56d16144-6aaa-4029-9428-bfb1fb676dbb","Type":"ContainerStarted","Data":"f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe"}
Oct 30 19:05:41.941600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.941476    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj" event={"ID":"56d16144-6aaa-4029-9428-bfb1fb676dbb","Type":"ContainerStarted","Data":"8f22b9d01c04c74b2139408fcd5920bf71db3bbaa810237bb048addc0961df26"}
Oct 30 19:05:41.941600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:41.941496    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:42.059333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:42.059236    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podStartSLOduration=2.059222865 podStartE2EDuration="2.059222865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:42.057897586 +0000 UTC m=+1134.642380587" watchObservedRunningTime="2026-10-30 19:05:42.059222865 +0000 UTC m=+1134.643705859"
Oct 30 19:05:42.171012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:42.170958    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj" podStartSLOduration=2.170944621 podStartE2EDuration="2.170944621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:42.170891479 +0000 UTC m=+1134.755374475" watchObservedRunningTime="2026-10-30 19:05:42.170944621 +0000 UTC m=+1134.755427618"
Oct 30 19:05:42.285269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:42.285228    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:42.286744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:42.286717    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:05:42.286953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:42.286774    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:05:43.285499 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:43.285465    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:05:43.285952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:43.285534    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:05:44.285731 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:44.285696    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:05:44.286137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:44.285753    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:05:45.285898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:45.285855    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:05:45.286306 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:45.285919    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:05:46.188637 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:05:46.188598    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:05:46.285543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:46.285504    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:05:46.285716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:46.285561    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:05:47.285665 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:47.285626    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:05:47.286084 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:47.285689    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:05:48.407834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:48.407563    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:48.411832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:48.408870    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:48.421722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:48.421698    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:05:52.970166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:52.970136    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:05:56.761160 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.761126    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"]
Oct 30 19:05:56.766099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.766076    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.808179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.808145    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpdhr\" (UniqueName: \"kubernetes.io/projected/5414afc5-892c-4b1c-a252-69980c82a4d9-kube-api-access-dpdhr\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.808329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.808203    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5414afc5-892c-4b1c-a252-69980c82a4d9-tls-certs\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.808329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.808311    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-home\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.808403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.808347    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-model-cache\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.808441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.808400    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-tmp-dir\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.808593 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.808571    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-dshm\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.863729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.863699    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"e2e-conflict-a-e7809b6e-kserve-self-signed-certs\""
Oct 30 19:05:56.866315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.866293    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"]
Oct 30 19:05:56.910375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.910334    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-dshm\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.911010 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.910645    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpdhr\" (UniqueName: \"kubernetes.io/projected/5414afc5-892c-4b1c-a252-69980c82a4d9-kube-api-access-dpdhr\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.911010 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.910710    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5414afc5-892c-4b1c-a252-69980c82a4d9-tls-certs\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.911010 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.910745    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-home\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.911010 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.910764    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-model-cache\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.911366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.910838    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-tmp-dir\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.911692 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.911668    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-tmp-dir\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.911921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.911883    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-home\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.912201 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.912182    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-model-cache\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.915284 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.915259    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5414afc5-892c-4b1c-a252-69980c82a4d9-tls-certs\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.916251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.916231    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-dshm\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:56.970537 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:56.970505    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpdhr\" (UniqueName: \"kubernetes.io/projected/5414afc5-892c-4b1c-a252-69980c82a4d9-kube-api-access-dpdhr\") pod \"e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:57.078100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:57.078008    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:57.469943 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:57.469899    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"]
Oct 30 19:05:57.476989 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:05:57.476952    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5414afc5_892c_4b1c_a252_69980c82a4d9.slice/crio-941f9c43c323d306232f8547c44d25b0f9ee01f7f0c20f8b4d21c5f0777e80d8 WatchSource:0}: Error finding container 941f9c43c323d306232f8547c44d25b0f9ee01f7f0c20f8b4d21c5f0777e80d8: Status 404 returned error can't find the container with id 941f9c43c323d306232f8547c44d25b0f9ee01f7f0c20f8b4d21c5f0777e80d8
Oct 30 19:05:58.060303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:58.060252    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894" event={"ID":"5414afc5-892c-4b1c-a252-69980c82a4d9","Type":"ContainerStarted","Data":"b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd"}
Oct 30 19:05:58.060303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:58.060307    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894" event={"ID":"5414afc5-892c-4b1c-a252-69980c82a4d9","Type":"ContainerStarted","Data":"941f9c43c323d306232f8547c44d25b0f9ee01f7f0c20f8b4d21c5f0777e80d8"}
Oct 30 19:05:58.078553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:58.078520    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:05:58.174200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:05:58.174147    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894" podStartSLOduration=2.174133226 podStartE2EDuration="2.174133226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:58.173125976 +0000 UTC m=+1150.757608973" watchObservedRunningTime="2026-10-30 19:05:58.174133226 +0000 UTC m=+1150.758616246"
Oct 30 19:06:00.058459 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.058422    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"]
Oct 30 19:06:00.062529 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.062506    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.145710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.145658    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6h94\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-kube-api-access-f6h94\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.145939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.145731    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.145939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.145857    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.145939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.145912    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.146086 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.145998    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.146129 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.146104    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.146182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.146163    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.146232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.146202    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.146461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.146434    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.169474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.169396    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"]
Oct 30 19:06:00.248175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248137    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248208    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6h94\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-kube-api-access-f6h94\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248254    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248288    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248334    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248367    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248403    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248449    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248481    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.248795 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.248772    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.249073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.249024    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.249168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.249126    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.249168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.249153    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.249964 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.249932    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.251148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.251122    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.251334 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.251311    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.308418 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.308378    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.308585 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.308417    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6h94\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-kube-api-access-f6h94\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-9j7nn\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.382690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.382651    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:00.810974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.810940    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"]
Oct 30 19:06:00.815784 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:06:00.815750    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1312ab0_a901_44e5_84ff_0a59ec6b59a5.slice/crio-d1b663cbfb22d4521b65db717cd084483f2c118faaeb34f7eec22d0e4c8bd87b WatchSource:0}: Error finding container d1b663cbfb22d4521b65db717cd084483f2c118faaeb34f7eec22d0e4c8bd87b: Status 404 returned error can't find the container with id d1b663cbfb22d4521b65db717cd084483f2c118faaeb34f7eec22d0e4c8bd87b
Oct 30 19:06:00.818160 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.818126    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:00.818264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.818199    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:00.818264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:00.818230    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:01.097024 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:01.096986    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" event={"ID":"c1312ab0-a901-44e5-84ff-0a59ec6b59a5","Type":"ContainerStarted","Data":"144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764"}
Oct 30 19:06:01.097436 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:01.097052    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" event={"ID":"c1312ab0-a901-44e5-84ff-0a59ec6b59a5","Type":"ContainerStarted","Data":"d1b663cbfb22d4521b65db717cd084483f2c118faaeb34f7eec22d0e4c8bd87b"}
Oct 30 19:06:01.216588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:01.216532    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podStartSLOduration=1.216518113 podStartE2EDuration="1.216518113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:01.214322082 +0000 UTC m=+1153.798805079" watchObservedRunningTime="2026-10-30 19:06:01.216518113 +0000 UTC m=+1153.801001109"
Oct 30 19:06:01.383550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:01.383458    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:01.385434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:01.385401    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:06:01.385556 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:01.385454    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:06:02.383842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:02.383782    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:06:02.384255 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:02.383860    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:06:03.383988 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.383952    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:06:03.384397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.384024    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:06:03.698730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.698647    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"]
Oct 30 19:06:03.703012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.702986    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.790038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.790001    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-routing-overlay-volume\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.790038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.790038    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5vxm\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-kube-api-access-c5vxm\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.790324 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.790096    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-plugins-config-volume\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.790324 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.790126    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-provider-credentials\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.805410 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.805374    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"]
Oct 30 19:06:03.891267 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.891231    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-plugins-config-volume\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.891452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.891283    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-provider-credentials\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.891452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.891364    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-routing-overlay-volume\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.891452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.891384    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5vxm\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-kube-api-access-c5vxm\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.892103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.892076    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-plugins-config-volume\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.892716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.892678    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-routing-overlay-volume\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.901452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.901422    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-provider-credentials\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:03.966924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:03.966829    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5vxm\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-kube-api-access-c5vxm\") pod \"payload-processing-external-model-589bbdb969-rrh4r\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") " pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:04.017636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:04.017596    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:04.383915 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:04.383872    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:06:04.384095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:04.383945    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:06:04.430129 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:04.430077    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"]
Oct 30 19:06:04.434282 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:06:04.434253    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7378665d_6e31_4c8b_80bd_4ae5dde510a2.slice/crio-3c646275f8b9aa20e248fba22cac334be2e70ae05c8fa05a8d7d51c920843619 WatchSource:0}: Error finding container 3c646275f8b9aa20e248fba22cac334be2e70ae05c8fa05a8d7d51c920843619: Status 404 returned error can't find the container with id 3c646275f8b9aa20e248fba22cac334be2e70ae05c8fa05a8d7d51c920843619
Oct 30 19:06:05.127905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:05.127868    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" event={"ID":"7378665d-6e31-4c8b-80bd-4ae5dde510a2","Type":"ContainerStarted","Data":"7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51"}
Oct 30 19:06:05.128120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:05.127930    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" event={"ID":"7378665d-6e31-4c8b-80bd-4ae5dde510a2","Type":"ContainerStarted","Data":"3c646275f8b9aa20e248fba22cac334be2e70ae05c8fa05a8d7d51c920843619"}
Oct 30 19:06:05.128120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:05.127956    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:05.262320 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:05.262259    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" podStartSLOduration=2.262242603 podStartE2EDuration="2.262242603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:05.240098418 +0000 UTC m=+1157.824581416" watchObservedRunningTime="2026-10-30 19:06:05.262242603 +0000 UTC m=+1157.846725599"
Oct 30 19:06:05.383072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:05.382987    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:06:05.383072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:05.383054    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:06:06.383998 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:06.383962    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:06:06.384589 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:06.384544    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:06:07.078669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:07.078636    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:06:07.097329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:07.097295    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:06:07.152574 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:07.152539    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:06:07.383878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:07.383843    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:06:07.384072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:07.383901    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:06:08.383673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:08.383633    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:06:08.383898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:08.383703    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:06:09.466085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:09.466048    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:09.466678 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:09.466598    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:09.467551 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:09.467527    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:06:15.199574 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.199534    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"]
Oct 30 19:06:15.204853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.204808    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.346636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.346599    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.346883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.346858    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52f2n\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-kube-api-access-52f2n\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.347074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.347051    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.347155 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.347083    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.347155 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.347113    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.347155 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.347136    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.347316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.347173    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.347316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.347214    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.347316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.347276    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.409950 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.409911    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"]
Oct 30 19:06:15.448038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448002    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448067    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52f2n\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-kube-api-access-52f2n\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448177    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448202    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448223    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448237    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448268    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448657 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448632    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448741 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448672    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448808 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448744    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.448892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448858    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.449009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.448988    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.449123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.449105    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.449643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.449592    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.450975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.450955    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.451099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.451078    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.507013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.506978    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.507182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.507160    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52f2n\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-kube-api-access-52f2n\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-7wr8s\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.519013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.518987    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:15.946474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.946431    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"]
Oct 30 19:06:15.951383 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:06:15.951350    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9a17d9_50a6_4063_8ae9_9690fdfd4df4.slice/crio-1f8d7d4e2dbbba74edbdcb0a8615f022449e5602d61f77dd00536d0cb2bcf031 WatchSource:0}: Error finding container 1f8d7d4e2dbbba74edbdcb0a8615f022449e5602d61f77dd00536d0cb2bcf031: Status 404 returned error can't find the container with id 1f8d7d4e2dbbba74edbdcb0a8615f022449e5602d61f77dd00536d0cb2bcf031
Oct 30 19:06:15.953794 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.953759    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:15.953894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.953842    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:15.953894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:15.953877    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:16.137601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:16.137557    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" podUID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:06:16.188867 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:06:16.188808    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:06:16.215586 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:16.215502    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" event={"ID":"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4","Type":"ContainerStarted","Data":"9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40"}
Oct 30 19:06:16.215586 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:16.215577    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" event={"ID":"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4","Type":"ContainerStarted","Data":"1f8d7d4e2dbbba74edbdcb0a8615f022449e5602d61f77dd00536d0cb2bcf031"}
Oct 30 19:06:16.334018 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:16.333963    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podStartSLOduration=1.333946232 podStartE2EDuration="1.333946232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:16.331905589 +0000 UTC m=+1168.916388588" watchObservedRunningTime="2026-10-30 19:06:16.333946232 +0000 UTC m=+1168.918429230"
Oct 30 19:06:16.519958 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:16.519867    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:17.521421 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:17.521383    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:06:17.521838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:17.521465    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:06:18.520166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:18.520129    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:06:18.520355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:18.520197    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:06:19.520383 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:19.520346    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:06:19.520799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:19.520415    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:06:20.520222 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:20.520179    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:06:20.520384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:20.520251    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:06:21.519982 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:21.519943    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:06:21.520420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:21.520012    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:06:22.520446 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:22.520404    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:06:22.520933 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:22.520487    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:06:23.519872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:23.519789    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:06:23.520052 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:23.519900    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:06:24.524565 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:24.524534    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:24.525000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:24.524827    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:24.525580 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:24.525561    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:06:26.135646 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:26.135605    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" podUID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:06:34.985941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:34.985906    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"]
Oct 30 19:06:34.987638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:34.987609    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" podUID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:06:35.024031 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.023993    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"]
Oct 30 19:06:35.027921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.027897    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.100777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.100742    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"]
Oct 30 19:06:35.161616 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.161575    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-routing-overlay-volume\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.161790 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.161637    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-provider-credentials\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.161790 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.161674    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-plugins-config-volume\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.161790 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.161766    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sph6m\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-kube-api-access-sph6m\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.262973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.262880    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-routing-overlay-volume\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.262973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.262933    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-provider-credentials\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.263200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.263054    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-plugins-config-volume\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.263200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.263140    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sph6m\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-kube-api-access-sph6m\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.263661 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.263639    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-routing-overlay-volume\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.263787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.263768    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-plugins-config-volume\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.265349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.265331    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-provider-credentials\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.322552 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.322513    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sph6m\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-kube-api-access-sph6m\") pod \"payload-processing-external-model-77f8b78bb8-fp6mm\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.341626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.341595    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:35.716465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:35.716428    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"]
Oct 30 19:06:35.721928 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:06:35.721889    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24f574c9_ada6_4dc9_b0ca_bb53d1d8cdf7.slice/crio-d50c8473c6366b113e29f9d0f4a8ad2e35d5ce1754657ca348d7cedc4cd2307f WatchSource:0}: Error finding container d50c8473c6366b113e29f9d0f4a8ad2e35d5ce1754657ca348d7cedc4cd2307f: Status 404 returned error can't find the container with id d50c8473c6366b113e29f9d0f4a8ad2e35d5ce1754657ca348d7cedc4cd2307f
Oct 30 19:06:36.374382 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:36.374342    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" event={"ID":"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7","Type":"ContainerStarted","Data":"57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24"}
Oct 30 19:06:36.374855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:36.374504    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:06:36.374855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:36.374538    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" event={"ID":"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7","Type":"ContainerStarted","Data":"d50c8473c6366b113e29f9d0f4a8ad2e35d5ce1754657ca348d7cedc4cd2307f"}
Oct 30 19:06:36.488610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:36.488555    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" podStartSLOduration=2.488541338 podStartE2EDuration="2.488541338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:36.486707398 +0000 UTC m=+1189.071190397" watchObservedRunningTime="2026-10-30 19:06:36.488541338 +0000 UTC m=+1189.073024334"
Oct 30 19:06:39.986802 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:39.986754    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" podUID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" containerName="payload-processing" containerID="cri-o://7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51" gracePeriod=25
Oct 30 19:06:40.257628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.257601    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:40.321513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.321471    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-kube-api-access-c5vxm\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-kube-api-access-c5vxm\") pod \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") "
Oct 30 19:06:40.321699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.321596    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-routing-overlay-volume\") pod \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") "
Oct 30 19:06:40.321699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.321625    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-plugins-config-volume\") pod \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") "
Oct 30 19:06:40.321699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.321669    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-provider-credentials\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-provider-credentials\") pod \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\" (UID: \"7378665d-6e31-4c8b-80bd-4ae5dde510a2\") "
Oct 30 19:06:40.322060 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.322035    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-routing-overlay-volume" pod "7378665d-6e31-4c8b-80bd-4ae5dde510a2" (UID: "7378665d-6e31-4c8b-80bd-4ae5dde510a2"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:40.322140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.322075    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-plugins-config-volume" pod "7378665d-6e31-4c8b-80bd-4ae5dde510a2" (UID: "7378665d-6e31-4c8b-80bd-4ae5dde510a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:40.322192 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.322177    5232 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-routing-overlay-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:40.322231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.322195    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7378665d-6e31-4c8b-80bd-4ae5dde510a2-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:40.324046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.324024    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-kube-api-access-c5vxm" pod "7378665d-6e31-4c8b-80bd-4ae5dde510a2" (UID: "7378665d-6e31-4c8b-80bd-4ae5dde510a2"). InnerVolumeSpecName "kube-api-access-c5vxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:40.324114 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.324056    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-provider-credentials" pod "7378665d-6e31-4c8b-80bd-4ae5dde510a2" (UID: "7378665d-6e31-4c8b-80bd-4ae5dde510a2"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:40.406754 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.406710    5232 generic.go:356] "Generic (PLEG): container finished" podID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" containerID="7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51" exitCode=0
Oct 30 19:06:40.406973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.406836    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"
Oct 30 19:06:40.406973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.406844    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" event={"ID":"7378665d-6e31-4c8b-80bd-4ae5dde510a2","Type":"ContainerDied","Data":"7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51"}
Oct 30 19:06:40.406973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.406904    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r" event={"ID":"7378665d-6e31-4c8b-80bd-4ae5dde510a2","Type":"ContainerDied","Data":"3c646275f8b9aa20e248fba22cac334be2e70ae05c8fa05a8d7d51c920843619"}
Oct 30 19:06:40.406973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.406928    5232 scope.go:122] "RemoveContainer" containerID="7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51"
Oct 30 19:06:40.418334 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.418317    5232 scope.go:122] "RemoveContainer" containerID="7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51"
Oct 30 19:06:40.418629 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:06:40.418610    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51\": container with ID starting with 7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51 not found: ID does not exist" containerID="7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51"
Oct 30 19:06:40.418702 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.418643    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51"} err="failed to get container status \"7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51\": rpc error: code = NotFound desc = could not find container \"7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51\": container with ID starting with 7ce910c36ed3164ee1a3f7b87aa62d109504b7cfd0a06c2de888aec03c6eee51 not found: ID does not exist"
Oct 30 19:06:40.423287 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.423266    5232 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-provider-credentials\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:40.423366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.423289    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5vxm\" (UniqueName: \"kubernetes.io/projected/7378665d-6e31-4c8b-80bd-4ae5dde510a2-kube-api-access-c5vxm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:40.565084 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.565001    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"]
Oct 30 19:06:40.569462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:40.569434    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-589bbdb969-rrh4r"]
Oct 30 19:06:41.539975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:41.539944    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" path="/var/lib/kubelet/pods/7378665d-6e31-4c8b-80bd-4ae5dde510a2/volumes"
Oct 30 19:06:46.188056 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:06:46.188011    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:06:47.390725 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:47.390689    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:06:47.669905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:47.668800    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:06:47.673928 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:47.673897    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:06:47.679186 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:47.679163    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:06:47.685367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:47.685334    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:06:47.766908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:47.766878    5232 scope.go:122] "RemoveContainer" containerID="9a3d8f215c2dda4523205e7b1548a8b821331acb7e0641fcd48180bbe101c70f"
Oct 30 19:06:50.266949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.266902    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"]
Oct 30 19:06:50.267568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.267554    5232 cpu_manager.go:414] "containerMap: removing container" podUID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" containerName="payload-processing"
Oct 30 19:06:50.267637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.267571    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" containerName="payload-processing"
Oct 30 19:06:50.267757 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.267742    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="7378665d-6e31-4c8b-80bd-4ae5dde510a2" containerName="payload-processing"
Oct 30 19:06:50.272510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.272489    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.366802 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.366757    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzpn9\" (UniqueName: \"kubernetes.io/projected/25b13db7-ee85-4e81-88f4-d5e2cb915640-kube-api-access-rzpn9\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.366802 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.366806    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-model-cache\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.367072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.366946    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25b13db7-ee85-4e81-88f4-d5e2cb915640-tls-certs\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.367164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.367134    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-tmp-dir\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.367245 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.367225    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-home\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.367309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.367261    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-dshm\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.370476 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.370447    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"e2e-conflict-b-e7809b6e-kserve-self-signed-certs\""
Oct 30 19:06:50.373348 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.373323    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"]
Oct 30 19:06:50.468511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.468470    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25b13db7-ee85-4e81-88f4-d5e2cb915640-tls-certs\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.468720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.468577    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-tmp-dir\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.468720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.468617    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-home\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.468720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.468638    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-dshm\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.468720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.468696    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzpn9\" (UniqueName: \"kubernetes.io/projected/25b13db7-ee85-4e81-88f4-d5e2cb915640-kube-api-access-rzpn9\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.468981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.468781    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-model-cache\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.469073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.469047    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-tmp-dir\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.469194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.469172    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-model-cache\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.469328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.469298    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-home\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.471076 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.471051    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-dshm\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.471358 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.471337    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25b13db7-ee85-4e81-88f4-d5e2cb915640-tls-certs\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.529822 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.529737    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzpn9\" (UniqueName: \"kubernetes.io/projected/25b13db7-ee85-4e81-88f4-d5e2cb915640-kube-api-access-rzpn9\") pod \"e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") " pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.587082 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.587046    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:50.995663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:50.995607    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"]
Oct 30 19:06:51.001008 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:06:51.000975    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25b13db7_ee85_4e81_88f4_d5e2cb915640.slice/crio-158b1667116f5c555509753aef0335d45183cf273e00c5156c3b46fac135f5d5 WatchSource:0}: Error finding container 158b1667116f5c555509753aef0335d45183cf273e00c5156c3b46fac135f5d5: Status 404 returned error can't find the container with id 158b1667116f5c555509753aef0335d45183cf273e00c5156c3b46fac135f5d5
Oct 30 19:06:51.484000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:51.483956    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5" event={"ID":"25b13db7-ee85-4e81-88f4-d5e2cb915640","Type":"ContainerStarted","Data":"136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f"}
Oct 30 19:06:51.484437 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:51.484008    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5" event={"ID":"25b13db7-ee85-4e81-88f4-d5e2cb915640","Type":"ContainerStarted","Data":"158b1667116f5c555509753aef0335d45183cf273e00c5156c3b46fac135f5d5"}
Oct 30 19:06:51.587911 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:51.587871    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:06:51.597835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:51.597755    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5" podStartSLOduration=1.597743186 podStartE2EDuration="1.597743186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:51.597236296 +0000 UTC m=+1204.181719293" watchObservedRunningTime="2026-10-30 19:06:51.597743186 +0000 UTC m=+1204.182226161"
Oct 30 19:06:57.388187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:06:57.388145    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:07:00.588098 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:00.588059    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:07:00.600234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:00.600205    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:07:01.562074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:01.562041    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:07:06.674463 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:06.674429    5232 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-c159be30ecbc: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:06.674463 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:06.674459    5232 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:06.675108 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:06.674532    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials podName:56d16144-6aaa-4029-9428-bfb1fb676dbb nodeName:}" failed. No retries permitted until 2026-10-30 19:07:07.17451589 +0000 UTC m=+1219.758998882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials") pod "payload-processing-external-model-5469ccd947-b2lgj" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb") : secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:07.181155 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:07.181122    5232 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-c159be30ecbc: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:07.181155 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:07.181150    5232 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:07.181377 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:07.181217    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials podName:56d16144-6aaa-4029-9428-bfb1fb676dbb nodeName:}" failed. No retries permitted until 2026-10-30 19:07:08.181202388 +0000 UTC m=+1220.765685394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials") pod "payload-processing-external-model-5469ccd947-b2lgj" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb") : secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:07.388094 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:07.388050    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:07:08.192391 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:08.192355    5232 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-c159be30ecbc: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:08.192391 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:08.192384    5232 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:08.192939 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:08.192447    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials podName:56d16144-6aaa-4029-9428-bfb1fb676dbb nodeName:}" failed. No retries permitted until 2026-10-30 19:07:10.192428515 +0000 UTC m=+1222.776911490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials") pod "payload-processing-external-model-5469ccd947-b2lgj" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb") : secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:10.213629 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:10.213593    5232 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-c159be30ecbc: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:10.213629 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:10.213626    5232 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:10.214116 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:10.213704    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials podName:56d16144-6aaa-4029-9428-bfb1fb676dbb nodeName:}" failed. No retries permitted until 2026-10-30 19:07:14.213685602 +0000 UTC m=+1226.798168582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials") pod "payload-processing-external-model-5469ccd947-b2lgj" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb") : secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:14.276088 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:14.276046    5232 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-c159be30ecbc: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:14.276088 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:14.276078    5232 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:14.276552 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:14.276170    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials podName:56d16144-6aaa-4029-9428-bfb1fb676dbb nodeName:}" failed. No retries permitted until 2026-10-30 19:07:22.276153749 +0000 UTC m=+1234.860636729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials") pod "payload-processing-external-model-5469ccd947-b2lgj" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb") : secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:14.794853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:14.794799    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"]
Oct 30 19:07:14.796355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:14.796323    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:07:16.188705 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:16.188667    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:07:19.795899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:19.795833    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerName="payload-processing" containerID="cri-o://57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24" gracePeriod=25
Oct 30 19:07:20.058532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.058463    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:07:20.144854 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.144801    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-routing-overlay-volume\") pod \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") "
Oct 30 19:07:20.145019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.144933    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-plugins-config-volume\") pod \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") "
Oct 30 19:07:20.145019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.144996    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-kube-api-access-sph6m\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-kube-api-access-sph6m\") pod \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") "
Oct 30 19:07:20.145156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.145047    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-provider-credentials\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-provider-credentials\") pod \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\" (UID: \"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7\") "
Oct 30 19:07:20.145210 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.145162    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-routing-overlay-volume" pod "24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" (UID: "24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:20.145304 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.145277    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-plugins-config-volume" pod "24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" (UID: "24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:20.145514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.145484    5232 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-routing-overlay-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:20.145514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.145503    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:20.147092 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.147058    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-kube-api-access-sph6m" pod "24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" (UID: "24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7"). InnerVolumeSpecName "kube-api-access-sph6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:20.147184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.147139    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-provider-credentials" pod "24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" (UID: "24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:20.246874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.246805    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sph6m\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-kube-api-access-sph6m\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:20.246874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.246873    5232 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7-provider-credentials\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:20.668211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.668165    5232 generic.go:356] "Generic (PLEG): container finished" podID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerID="57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24" exitCode=0
Oct 30 19:07:20.668211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.668210    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" event={"ID":"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7","Type":"ContainerDied","Data":"57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24"}
Oct 30 19:07:20.668481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.668216    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"
Oct 30 19:07:20.668481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.668262    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm" event={"ID":"24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7","Type":"ContainerDied","Data":"d50c8473c6366b113e29f9d0f4a8ad2e35d5ce1754657ca348d7cedc4cd2307f"}
Oct 30 19:07:20.668481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.668290    5232 scope.go:122] "RemoveContainer" containerID="57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24"
Oct 30 19:07:20.679473 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.679453    5232 scope.go:122] "RemoveContainer" containerID="57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24"
Oct 30 19:07:20.679772 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:20.679749    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24\": container with ID starting with 57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24 not found: ID does not exist" containerID="57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24"
Oct 30 19:07:20.679961 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.679783    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24"} err="failed to get container status \"57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24\": rpc error: code = NotFound desc = could not find container \"57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24\": container with ID starting with 57e3a779750d00f3dc0b888af5036ed7b37019db8287c6236691d3b951772d24 not found: ID does not exist"
Oct 30 19:07:20.826617 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.826582    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"]
Oct 30 19:07:20.830488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:20.830455    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-fp6mm"]
Oct 30 19:07:21.539271 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:21.539237    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" path="/var/lib/kubelet/pods/24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7/volumes"
Oct 30 19:07:22.370835 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:22.370723    5232 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-c159be30ecbc: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:22.370835 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:22.370760    5232 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj: secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:22.370835 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:22.370842    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials podName:56d16144-6aaa-4029-9428-bfb1fb676dbb nodeName:}" failed. No retries permitted until 2026-10-30 19:07:38.37080342 +0000 UTC m=+1250.955286410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials") pod "payload-processing-external-model-5469ccd947-b2lgj" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb") : secret "agc-cred-c159be30ecbc" not found
Oct 30 19:07:25.513023 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:25.512979    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:07:25.513669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:25.513637    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:07:25.568846 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:25.567969    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:07:25.568846 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:25.568058    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:07:27.209979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:27.209942    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"]
Oct 30 19:07:32.210432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.210381    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj" podUID="56d16144-6aaa-4029-9428-bfb1fb676dbb" containerName="payload-processing" containerID="cri-o://f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe" gracePeriod=25
Oct 30 19:07:32.520049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.520018    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:07:32.616175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.616134    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials\") pod \"56d16144-6aaa-4029-9428-bfb1fb676dbb\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") "
Oct 30 19:07:32.616374 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.616294    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-routing-overlay-volume\") pod \"56d16144-6aaa-4029-9428-bfb1fb676dbb\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") "
Oct 30 19:07:32.616432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.616409    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-plugins-config-volume\") pod \"56d16144-6aaa-4029-9428-bfb1fb676dbb\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") "
Oct 30 19:07:32.616485 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.616441    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-kube-api-access-wkc6k\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-kube-api-access-wkc6k\") pod \"56d16144-6aaa-4029-9428-bfb1fb676dbb\" (UID: \"56d16144-6aaa-4029-9428-bfb1fb676dbb\") "
Oct 30 19:07:32.617254 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.617005    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-routing-overlay-volume" pod "56d16144-6aaa-4029-9428-bfb1fb676dbb" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:32.617732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.617685    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-plugins-config-volume" pod "56d16144-6aaa-4029-9428-bfb1fb676dbb" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:32.620890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.620318    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials" pod "56d16144-6aaa-4029-9428-bfb1fb676dbb" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:32.620890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.620437    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-kube-api-access-wkc6k" pod "56d16144-6aaa-4029-9428-bfb1fb676dbb" (UID: "56d16144-6aaa-4029-9428-bfb1fb676dbb"). InnerVolumeSpecName "kube-api-access-wkc6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:32.720852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.720774    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:32.720852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.720843    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkc6k\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-kube-api-access-wkc6k\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:32.721119 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.720861    5232 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/56d16144-6aaa-4029-9428-bfb1fb676dbb-provider-credentials\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:32.721119 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.720877    5232 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/56d16144-6aaa-4029-9428-bfb1fb676dbb-routing-overlay-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:32.828983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.828894    5232 generic.go:356] "Generic (PLEG): container finished" podID="56d16144-6aaa-4029-9428-bfb1fb676dbb" containerID="f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe" exitCode=0
Oct 30 19:07:32.828983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.828939    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"
Oct 30 19:07:32.828983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.828970    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj" event={"ID":"56d16144-6aaa-4029-9428-bfb1fb676dbb","Type":"ContainerDied","Data":"f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe"}
Oct 30 19:07:32.829230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.829027    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj" event={"ID":"56d16144-6aaa-4029-9428-bfb1fb676dbb","Type":"ContainerDied","Data":"8f22b9d01c04c74b2139408fcd5920bf71db3bbaa810237bb048addc0961df26"}
Oct 30 19:07:32.829230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.829058    5232 scope.go:122] "RemoveContainer" containerID="f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe"
Oct 30 19:07:32.904012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.898210    5232 scope.go:122] "RemoveContainer" containerID="f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe"
Oct 30 19:07:32.904012 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:32.898626    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe\": container with ID starting with f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe not found: ID does not exist" containerID="f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe"
Oct 30 19:07:32.904012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:32.898666    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe"} err="failed to get container status \"f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe\": rpc error: code = NotFound desc = could not find container \"f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe\": container with ID starting with f8e92973e9c75774856044799e0a79b0629f9244ab8e7bd8dcee8607e990c6fe not found: ID does not exist"
Oct 30 19:07:32.935513 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:32.935448    5232 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d16144_6aaa_4029_9428_bfb1fb676dbb.slice/crio-8f22b9d01c04c74b2139408fcd5920bf71db3bbaa810237bb048addc0961df26\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d16144_6aaa_4029_9428_bfb1fb676dbb.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 19:07:33.013836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:33.010204    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"]
Oct 30 19:07:33.013836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:33.013148    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-5469ccd947-b2lgj"]
Oct 30 19:07:33.558843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:33.557905    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56d16144-6aaa-4029-9428-bfb1fb676dbb" path="/var/lib/kubelet/pods/56d16144-6aaa-4029-9428-bfb1fb676dbb/volumes"
Oct 30 19:07:41.153706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.153671    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"]
Oct 30 19:07:41.156408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.154289    5232 cpu_manager.go:414] "containerMap: removing container" podUID="56d16144-6aaa-4029-9428-bfb1fb676dbb" containerName="payload-processing"
Oct 30 19:07:41.156408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.154305    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56d16144-6aaa-4029-9428-bfb1fb676dbb" containerName="payload-processing"
Oct 30 19:07:41.156408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.154322    5232 cpu_manager.go:414] "containerMap: removing container" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerName="payload-processing"
Oct 30 19:07:41.156408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.154327    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerName="payload-processing"
Oct 30 19:07:41.156408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.154418    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="56d16144-6aaa-4029-9428-bfb1fb676dbb" containerName="payload-processing"
Oct 30 19:07:41.156408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.154428    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="24f574c9-ada6-4dc9-b0ca-bb53d1d8cdf7" containerName="payload-processing"
Oct 30 19:07:41.157477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.157461    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.226323 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.226283    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b361acb-b72d-4eb1-8bd1-272e4399f509-tls-certs\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.226483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.226400    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-tmp-dir\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.226483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.226444    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-dshm\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.226578 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.226490    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-home\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.226628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.226587    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-model-cache\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.226683 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.226649    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzb6m\" (UniqueName: \"kubernetes.io/projected/1b361acb-b72d-4eb1-8bd1-272e4399f509-kube-api-access-xzb6m\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.255064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.255021    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-9275c3\"/\"openshift-service-ca.crt\""
Oct 30 19:07:41.255263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.255177    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-9275c3\"/\"default-dockercfg-7g5c2\""
Oct 30 19:07:41.255263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.255203    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-9275c3\"/\"distinct-w1-9275c3-kserve-self-signed-certs\""
Oct 30 19:07:41.255362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.255262    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-9275c3\"/\"kube-root-ca.crt\""
Oct 30 19:07:41.258022 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.258000    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"]
Oct 30 19:07:41.327618 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.327580    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-dshm\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.327830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.327645    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-home\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.327830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.327684    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-model-cache\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.327830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.327704    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzb6m\" (UniqueName: \"kubernetes.io/projected/1b361acb-b72d-4eb1-8bd1-272e4399f509-kube-api-access-xzb6m\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.328027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.327849    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b361acb-b72d-4eb1-8bd1-272e4399f509-tls-certs\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.328027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.327907    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-tmp-dir\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.328176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.328154    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-home\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.328243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.328199    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-model-cache\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.328243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.328216    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-tmp-dir\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.329938 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.329913    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-dshm\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.330194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.330176    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b361acb-b72d-4eb1-8bd1-272e4399f509-tls-certs\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.387404 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.387365    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzb6m\" (UniqueName: \"kubernetes.io/projected/1b361acb-b72d-4eb1-8bd1-272e4399f509-kube-api-access-xzb6m\") pod \"distinct-w1-9275c3-kserve-6db7d485d5-gbg2g\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.469224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.469136    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:41.865238 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.865201    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"]
Oct 30 19:07:41.889619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:41.889585    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" event={"ID":"1b361acb-b72d-4eb1-8bd1-272e4399f509","Type":"ContainerStarted","Data":"d8847a26e8eaaceb1482d5f737c05fbd1534d5e4ff4b2c05177633864d5c9c3c"}
Oct 30 19:07:42.900650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:42.900585    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" event={"ID":"1b361acb-b72d-4eb1-8bd1-272e4399f509","Type":"ContainerStarted","Data":"e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0"}
Oct 30 19:07:43.016699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:43.016642    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podStartSLOduration=2.016627179 podStartE2EDuration="2.016627179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:43.014663136 +0000 UTC m=+1255.599146132" watchObservedRunningTime="2026-10-30 19:07:43.016627179 +0000 UTC m=+1255.601110175"
Oct 30 19:07:43.469909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:43.469871    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:46.187997 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:07:46.187960    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:07:51.469627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:51.469591    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:51.482107 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:51.482080    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:07:51.973269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:07:51.973239    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:08:00.371781 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:00.371697    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"]
Oct 30 19:08:15.344264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.344220    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"]
Oct 30 19:08:15.348617 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.348593    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.391873 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.391797    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5" podUID="25b13db7-ee85-4e81-88f4-d5e2cb915640" containerName="main" containerID="cri-o://136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f" gracePeriod=45
Oct 30 19:08:15.398922 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.398888    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-plugins-config-volume\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.399037 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.398960    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-routing-overlay-volume\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.399091 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.399063    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-provider-credentials\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.399130 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.399071    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-7378d4fb8020\""
Oct 30 19:08:15.399269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.399251    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzvwn\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-kube-api-access-gzvwn\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.446761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.446728    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 19:08:15.446946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.446777    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 19:08:15.446946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.446866    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-bvv24\""
Oct 30 19:08:15.447070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.447051    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 19:08:15.447271 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.447255    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 19:08:15.451657 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.451633    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"]
Oct 30 19:08:15.500749 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.500715    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzvwn\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-kube-api-access-gzvwn\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.500949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.500796    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-plugins-config-volume\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.500949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.500871    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-routing-overlay-volume\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.500949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.500916    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-provider-credentials\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.504668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.501712    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-routing-overlay-volume\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.504668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.501730    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-plugins-config-volume\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.504668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.503917    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-provider-credentials\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.551239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.551195    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5" podUID="25b13db7-ee85-4e81-88f4-d5e2cb915640" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": dial tcp 10.133.0.84:8000: connect: connection refused"
Oct 30 19:08:15.560379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.560348    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzvwn\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-kube-api-access-gzvwn\") pod \"payload-processing-external-model-77f8b78bb8-257bw\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") " pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.663932 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.663900    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:15.679280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.679251    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:08:15.812835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.805208    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25b13db7-ee85-4e81-88f4-d5e2cb915640-kube-api-access-rzpn9\" (UniqueName: \"kubernetes.io/projected/25b13db7-ee85-4e81-88f4-d5e2cb915640-kube-api-access-rzpn9\") pod \"25b13db7-ee85-4e81-88f4-d5e2cb915640\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") "
Oct 30 19:08:15.812835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.805291    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-model-cache\") pod \"25b13db7-ee85-4e81-88f4-d5e2cb915640\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") "
Oct 30 19:08:15.812835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.805389    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-tmp-dir\") pod \"25b13db7-ee85-4e81-88f4-d5e2cb915640\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") "
Oct 30 19:08:15.812835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.805460    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-dshm\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-dshm\") pod \"25b13db7-ee85-4e81-88f4-d5e2cb915640\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") "
Oct 30 19:08:15.812835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.805572    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25b13db7-ee85-4e81-88f4-d5e2cb915640-tls-certs\" (UniqueName: \"kubernetes.io/secret/25b13db7-ee85-4e81-88f4-d5e2cb915640-tls-certs\") pod \"25b13db7-ee85-4e81-88f4-d5e2cb915640\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") "
Oct 30 19:08:15.812835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.805611    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-home\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-home\") pod \"25b13db7-ee85-4e81-88f4-d5e2cb915640\" (UID: \"25b13db7-ee85-4e81-88f4-d5e2cb915640\") "
Oct 30 19:08:15.812835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.806302    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-home" pod "25b13db7-ee85-4e81-88f4-d5e2cb915640" (UID: "25b13db7-ee85-4e81-88f4-d5e2cb915640"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:15.818032 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.817969    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-model-cache" pod "25b13db7-ee85-4e81-88f4-d5e2cb915640" (UID: "25b13db7-ee85-4e81-88f4-d5e2cb915640"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:15.854000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.818954    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-tmp-dir" pod "25b13db7-ee85-4e81-88f4-d5e2cb915640" (UID: "25b13db7-ee85-4e81-88f4-d5e2cb915640"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:15.854000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.840013    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25b13db7-ee85-4e81-88f4-d5e2cb915640-tls-certs" pod "25b13db7-ee85-4e81-88f4-d5e2cb915640" (UID: "25b13db7-ee85-4e81-88f4-d5e2cb915640"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:08:15.856085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.855976    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25b13db7-ee85-4e81-88f4-d5e2cb915640-kube-api-access-rzpn9" pod "25b13db7-ee85-4e81-88f4-d5e2cb915640" (UID: "25b13db7-ee85-4e81-88f4-d5e2cb915640"). InnerVolumeSpecName "kube-api-access-rzpn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:15.857729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.857696    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-dshm" pod "25b13db7-ee85-4e81-88f4-d5e2cb915640" (UID: "25b13db7-ee85-4e81-88f4-d5e2cb915640"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:15.907174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.907089    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzpn9\" (UniqueName: \"kubernetes.io/projected/25b13db7-ee85-4e81-88f4-d5e2cb915640-kube-api-access-rzpn9\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:15.907174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.907119    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:15.907174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.907129    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:15.907174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.907139    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:15.907174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.907147    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25b13db7-ee85-4e81-88f4-d5e2cb915640-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:15.907174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:15.907155    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25b13db7-ee85-4e81-88f4-d5e2cb915640-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:16.041855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.041093    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"]
Oct 30 19:08:16.046163 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:08:16.046129    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4adb0efa_0ba3_4c09_950e_c2d8f0f2c1f0.slice/crio-7c0836f8d3d83c9f234948f6e0e4d883ea58a8b5059b32769f85cf92544c1f59 WatchSource:0}: Error finding container 7c0836f8d3d83c9f234948f6e0e4d883ea58a8b5059b32769f85cf92544c1f59: Status 404 returned error can't find the container with id 7c0836f8d3d83c9f234948f6e0e4d883ea58a8b5059b32769f85cf92544c1f59
Oct 30 19:08:16.110116 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.110090    5232 generic.go:356] "Generic (PLEG): container finished" podID="25b13db7-ee85-4e81-88f4-d5e2cb915640" containerID="136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f" exitCode=0
Oct 30 19:08:16.110239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.110140    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"
Oct 30 19:08:16.110239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.110169    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5" event={"ID":"25b13db7-ee85-4e81-88f4-d5e2cb915640","Type":"ContainerDied","Data":"136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f"}
Oct 30 19:08:16.110239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.110214    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5" event={"ID":"25b13db7-ee85-4e81-88f4-d5e2cb915640","Type":"ContainerDied","Data":"158b1667116f5c555509753aef0335d45183cf273e00c5156c3b46fac135f5d5"}
Oct 30 19:08:16.110239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.110231    5232 scope.go:122] "RemoveContainer" containerID="136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f"
Oct 30 19:08:16.112245 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.112221    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw" event={"ID":"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0","Type":"ContainerStarted","Data":"7c0836f8d3d83c9f234948f6e0e4d883ea58a8b5059b32769f85cf92544c1f59"}
Oct 30 19:08:16.112366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.112319    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:16.120084 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.120065    5232 scope.go:122] "RemoveContainer" containerID="136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f"
Oct 30 19:08:16.120404 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:08:16.120381    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f\": container with ID starting with 136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f not found: ID does not exist" containerID="136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f"
Oct 30 19:08:16.120483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.120414    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f"} err="failed to get container status \"136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f\": rpc error: code = NotFound desc = could not find container \"136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f\": container with ID starting with 136ab9069bad6cb3452f3ad0861722de8deea24198790487fb5ef5bd75a2b13f not found: ID does not exist"
Oct 30 19:08:16.188940 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:08:16.188850    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:08:16.275572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.275519    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw" podStartSLOduration=1.27550469 podStartE2EDuration="1.27550469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:16.274329343 +0000 UTC m=+1288.858812340" watchObservedRunningTime="2026-10-30 19:08:16.27550469 +0000 UTC m=+1288.859987686"
Oct 30 19:08:16.383052 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.383007    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"]
Oct 30 19:08:16.388412 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:16.388382    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-b-e7809b6e-kserve-69d54b6c47-9tlp5"]
Oct 30 19:08:17.120346 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:17.120313    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw" event={"ID":"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0","Type":"ContainerStarted","Data":"46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d"}
Oct 30 19:08:17.541444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:17.541364    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25b13db7-ee85-4e81-88f4-d5e2cb915640" path="/var/lib/kubelet/pods/25b13db7-ee85-4e81-88f4-d5e2cb915640/volumes"
Oct 30 19:08:28.184836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:28.174935    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw" podUID="4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:08:28.487899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:28.485342    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"]
Oct 30 19:08:28.515856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:28.511915    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw" podUID="4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:08:33.486433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.486352    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw" podUID="4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" containerName="payload-processing" containerID="cri-o://46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d" gracePeriod=25
Oct 30 19:08:33.746841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.746755    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:33.859469 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.859437    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-routing-overlay-volume\") pod \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") "
Oct 30 19:08:33.859666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.859503    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-plugins-config-volume\") pod \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") "
Oct 30 19:08:33.859666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.859523    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-provider-credentials\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-provider-credentials\") pod \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") "
Oct 30 19:08:33.859666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.859576    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-kube-api-access-gzvwn\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-kube-api-access-gzvwn\") pod \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\" (UID: \"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0\") "
Oct 30 19:08:33.859963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.859932    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-routing-overlay-volume" pod "4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" (UID: "4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:33.860065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.860045    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-plugins-config-volume" pod "4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" (UID: "4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:33.861586 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.861565    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-provider-credentials" pod "4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" (UID: "4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:33.861665 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.861629    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-kube-api-access-gzvwn" pod "4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" (UID: "4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0"). InnerVolumeSpecName "kube-api-access-gzvwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:33.960653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.960614    5232 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-routing-overlay-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:33.960653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.960647    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:33.960653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.960657    5232 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-provider-credentials\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:33.960921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:33.960667    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzvwn\" (UniqueName: \"kubernetes.io/projected/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0-kube-api-access-gzvwn\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:34.358228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.358192    5232 generic.go:356] "Generic (PLEG): container finished" podID="4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" containerID="46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d" exitCode=0
Oct 30 19:08:34.358429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.358236    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"
Oct 30 19:08:34.358429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.358271    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw" event={"ID":"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0","Type":"ContainerDied","Data":"46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d"}
Oct 30 19:08:34.358429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.358323    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw" event={"ID":"4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0","Type":"ContainerDied","Data":"7c0836f8d3d83c9f234948f6e0e4d883ea58a8b5059b32769f85cf92544c1f59"}
Oct 30 19:08:34.358429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.358344    5232 scope.go:122] "RemoveContainer" containerID="46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d"
Oct 30 19:08:34.368737 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.368720    5232 scope.go:122] "RemoveContainer" containerID="46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d"
Oct 30 19:08:34.369011 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:08:34.368991    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d\": container with ID starting with 46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d not found: ID does not exist" containerID="46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d"
Oct 30 19:08:34.369082 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.369022    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d"} err="failed to get container status \"46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d\": rpc error: code = NotFound desc = could not find container \"46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d\": container with ID starting with 46887b2c32ed74d884319e7e90c946f31e545db8df5da1295ba613bc7b96295d not found: ID does not exist"
Oct 30 19:08:34.518215 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.518175    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"]
Oct 30 19:08:34.565338 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:34.565305    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-77f8b78bb8-257bw"]
Oct 30 19:08:35.539630 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:35.539599    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" path="/var/lib/kubelet/pods/4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0/volumes"
Oct 30 19:08:35.821821 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:35.821706    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"]
Oct 30 19:08:46.187267 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:08:46.187220    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:08:50.841733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:50.841664    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894" podUID="5414afc5-892c-4b1c-a252-69980c82a4d9" containerName="main" containerID="cri-o://b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd" gracePeriod=45
Oct 30 19:08:51.109794 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.109766    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:08:51.277648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.277615    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-tmp-dir\") pod \"5414afc5-892c-4b1c-a252-69980c82a4d9\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") "
Oct 30 19:08:51.277870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.277660    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5414afc5-892c-4b1c-a252-69980c82a4d9-tls-certs\" (UniqueName: \"kubernetes.io/secret/5414afc5-892c-4b1c-a252-69980c82a4d9-tls-certs\") pod \"5414afc5-892c-4b1c-a252-69980c82a4d9\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") "
Oct 30 19:08:51.277870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.277704    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5414afc5-892c-4b1c-a252-69980c82a4d9-kube-api-access-dpdhr\" (UniqueName: \"kubernetes.io/projected/5414afc5-892c-4b1c-a252-69980c82a4d9-kube-api-access-dpdhr\") pod \"5414afc5-892c-4b1c-a252-69980c82a4d9\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") "
Oct 30 19:08:51.277870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.277763    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-dshm\") pod \"5414afc5-892c-4b1c-a252-69980c82a4d9\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") "
Oct 30 19:08:51.277870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.277805    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-home\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-home\") pod \"5414afc5-892c-4b1c-a252-69980c82a4d9\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") "
Oct 30 19:08:51.277870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.277848    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-model-cache\") pod \"5414afc5-892c-4b1c-a252-69980c82a4d9\" (UID: \"5414afc5-892c-4b1c-a252-69980c82a4d9\") "
Oct 30 19:08:51.278151 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.277984    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-tmp-dir" pod "5414afc5-892c-4b1c-a252-69980c82a4d9" (UID: "5414afc5-892c-4b1c-a252-69980c82a4d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:51.278151 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.278113    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-home" pod "5414afc5-892c-4b1c-a252-69980c82a4d9" (UID: "5414afc5-892c-4b1c-a252-69980c82a4d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:51.278259 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.278159    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-model-cache" pod "5414afc5-892c-4b1c-a252-69980c82a4d9" (UID: "5414afc5-892c-4b1c-a252-69980c82a4d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:51.278323 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.278268    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:51.278323 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.278280    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:51.278323 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.278288    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:51.280011 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.279983    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-dshm" pod "5414afc5-892c-4b1c-a252-69980c82a4d9" (UID: "5414afc5-892c-4b1c-a252-69980c82a4d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:51.280011 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.279995    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5414afc5-892c-4b1c-a252-69980c82a4d9-tls-certs" pod "5414afc5-892c-4b1c-a252-69980c82a4d9" (UID: "5414afc5-892c-4b1c-a252-69980c82a4d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:08:51.280149 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.280038    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5414afc5-892c-4b1c-a252-69980c82a4d9-kube-api-access-dpdhr" pod "5414afc5-892c-4b1c-a252-69980c82a4d9" (UID: "5414afc5-892c-4b1c-a252-69980c82a4d9"). InnerVolumeSpecName "kube-api-access-dpdhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:51.379942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.379858    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5414afc5-892c-4b1c-a252-69980c82a4d9-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:51.379942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.379891    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpdhr\" (UniqueName: \"kubernetes.io/projected/5414afc5-892c-4b1c-a252-69980c82a4d9-kube-api-access-dpdhr\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:51.379942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.379906    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5414afc5-892c-4b1c-a252-69980c82a4d9-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:51.472081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.471841    5232 generic.go:356] "Generic (PLEG): container finished" podID="5414afc5-892c-4b1c-a252-69980c82a4d9" containerID="b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd" exitCode=0
Oct 30 19:08:51.472081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.471889    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"
Oct 30 19:08:51.472081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.471902    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894" event={"ID":"5414afc5-892c-4b1c-a252-69980c82a4d9","Type":"ContainerDied","Data":"b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd"}
Oct 30 19:08:51.472081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.471946    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894" event={"ID":"5414afc5-892c-4b1c-a252-69980c82a4d9","Type":"ContainerDied","Data":"941f9c43c323d306232f8547c44d25b0f9ee01f7f0c20f8b4d21c5f0777e80d8"}
Oct 30 19:08:51.472081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.471968    5232 scope.go:122] "RemoveContainer" containerID="b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd"
Oct 30 19:08:51.481891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.481857    5232 scope.go:122] "RemoveContainer" containerID="b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd"
Oct 30 19:08:51.482244 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:08:51.482216    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd\": container with ID starting with b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd not found: ID does not exist" containerID="b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd"
Oct 30 19:08:51.482358 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.482252    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd"} err="failed to get container status \"b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd\": rpc error: code = NotFound desc = could not find container \"b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd\": container with ID starting with b424802c77276d538b916378cb52da3e7cb0e94a1cdcb9e6dbd0e2494f5ac2fd not found: ID does not exist"
Oct 30 19:08:51.634453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.634362    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"]
Oct 30 19:08:51.637126 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:51.637099    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/e2e-conflict-a-e7809b6e-kserve-645fd79845-7n894"]
Oct 30 19:08:53.544744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.544707    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5414afc5-892c-4b1c-a252-69980c82a4d9" path="/var/lib/kubelet/pods/5414afc5-892c-4b1c-a252-69980c82a4d9/volumes"
Oct 30 19:08:53.582982 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.582947    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"]
Oct 30 19:08:53.583519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583499    5232 cpu_manager.go:414] "containerMap: removing container" podUID="25b13db7-ee85-4e81-88f4-d5e2cb915640" containerName="main"
Oct 30 19:08:53.583519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583518    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25b13db7-ee85-4e81-88f4-d5e2cb915640" containerName="main"
Oct 30 19:08:53.583690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583530    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5414afc5-892c-4b1c-a252-69980c82a4d9" containerName="main"
Oct 30 19:08:53.583690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583535    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5414afc5-892c-4b1c-a252-69980c82a4d9" containerName="main"
Oct 30 19:08:53.583690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583554    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" containerName="payload-processing"
Oct 30 19:08:53.583690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583561    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" containerName="payload-processing"
Oct 30 19:08:53.583690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583665    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="25b13db7-ee85-4e81-88f4-d5e2cb915640" containerName="main"
Oct 30 19:08:53.583690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583678    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5414afc5-892c-4b1c-a252-69980c82a4d9" containerName="main"
Oct 30 19:08:53.583690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.583689    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="4adb0efa-0ba3-4c09-950e-c2d8f0f2c1f0" containerName="payload-processing"
Oct 30 19:08:53.588499 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.588479    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.686362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.686330    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-22fb18\"/\"distinct-2-w3-22fb18-kserve-self-signed-certs\""
Oct 30 19:08:53.689443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.689419    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"]
Oct 30 19:08:53.704176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.704146    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp7mj\" (UniqueName: \"kubernetes.io/projected/52b5f834-e9aa-4dae-b573-226777de2af1-kube-api-access-zp7mj\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.704332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.704191    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-dshm\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.704332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.704223    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5f834-e9aa-4dae-b573-226777de2af1-tls-certs\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.704444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.704407    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-model-cache\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.704536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.704452    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-tmp-dir\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.704536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.704499    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-home\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.805774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.805683    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp7mj\" (UniqueName: \"kubernetes.io/projected/52b5f834-e9aa-4dae-b573-226777de2af1-kube-api-access-zp7mj\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.805774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.805734    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-dshm\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.806051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.805776    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5f834-e9aa-4dae-b573-226777de2af1-tls-certs\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.806051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.805800    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-model-cache\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.806051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.805870    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-tmp-dir\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.806051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.805909    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-home\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.806276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.806242    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-model-cache\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.806326 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.806296    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-tmp-dir\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.806326 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.806309    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-home\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.808211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.808184    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-dshm\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.808427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.808412    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5f834-e9aa-4dae-b573-226777de2af1-tls-certs\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.878227 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.878183    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp7mj\" (UniqueName: \"kubernetes.io/projected/52b5f834-e9aa-4dae-b573-226777de2af1-kube-api-access-zp7mj\") pod \"distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") " pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:53.900128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:53.900096    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:08:54.305645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:54.305598    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"]
Oct 30 19:08:54.310381 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:08:54.310356    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52b5f834_e9aa_4dae_b573_226777de2af1.slice/crio-c2385d6776c88d67abff86a3a7312ee25a7d16b65a984d81ce023468bc200fc9 WatchSource:0}: Error finding container c2385d6776c88d67abff86a3a7312ee25a7d16b65a984d81ce023468bc200fc9: Status 404 returned error can't find the container with id c2385d6776c88d67abff86a3a7312ee25a7d16b65a984d81ce023468bc200fc9
Oct 30 19:08:54.491419 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:54.491378    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" event={"ID":"52b5f834-e9aa-4dae-b573-226777de2af1","Type":"ContainerStarted","Data":"a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053"}
Oct 30 19:08:54.491419 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:54.491429    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" event={"ID":"52b5f834-e9aa-4dae-b573-226777de2af1","Type":"ContainerStarted","Data":"c2385d6776c88d67abff86a3a7312ee25a7d16b65a984d81ce023468bc200fc9"}
Oct 30 19:08:54.630842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:54.630770    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podStartSLOduration=1.630752822 podStartE2EDuration="1.630752822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:54.628025291 +0000 UTC m=+1327.212508288" watchObservedRunningTime="2026-10-30 19:08:54.630752822 +0000 UTC m=+1327.215235820"
Oct 30 19:08:54.901199 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:08:54.901122    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:09:02.722592 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.722549    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"]
Oct 30 19:09:02.727688 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.727657    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.779015 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.778981    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-9275c3\"/\"distinct-2-w1-9275c3-kserve-self-signed-certs\""
Oct 30 19:09:02.809973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.809933    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-home\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.810161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.810115    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tmp-dir\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.810219 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.810171    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r96l4\" (UniqueName: \"kubernetes.io/projected/38887470-2bd7-4df2-9aca-cb8d6bccec4a-kube-api-access-r96l4\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.810297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.810273    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-model-cache\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.810350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.810334    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tls-certs\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.810462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.810432    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-dshm\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.831273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.831231    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"]
Oct 30 19:09:02.911422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911383    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tmp-dir\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.911597 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911436    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r96l4\" (UniqueName: \"kubernetes.io/projected/38887470-2bd7-4df2-9aca-cb8d6bccec4a-kube-api-access-r96l4\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.911597 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911465    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-model-cache\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.911597 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911493    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tls-certs\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.911597 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911512    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-dshm\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.911597 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911561    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-home\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.911909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911886    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tmp-dir\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.911998 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911969    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-model-cache\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.912081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.911991    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-home\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.913807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.913782    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-dshm\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.914244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.914181    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tls-certs\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:02.973468 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:02.973388    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r96l4\" (UniqueName: \"kubernetes.io/projected/38887470-2bd7-4df2-9aca-cb8d6bccec4a-kube-api-access-r96l4\") pod \"distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") " pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:03.041797 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:03.041756    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:03.458843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:03.450535    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"]
Oct 30 19:09:03.458843 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:09:03.457178    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38887470_2bd7_4df2_9aca_cb8d6bccec4a.slice/crio-d9d62c11b7b5b25c5177b072b2a4321741de5f89d41ee15af98ede2c308a7605 WatchSource:0}: Error finding container d9d62c11b7b5b25c5177b072b2a4321741de5f89d41ee15af98ede2c308a7605: Status 404 returned error can't find the container with id d9d62c11b7b5b25c5177b072b2a4321741de5f89d41ee15af98ede2c308a7605
Oct 30 19:09:03.549904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:03.549860    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" event={"ID":"38887470-2bd7-4df2-9aca-cb8d6bccec4a","Type":"ContainerStarted","Data":"8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d"}
Oct 30 19:09:03.550224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:03.550199    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" event={"ID":"38887470-2bd7-4df2-9aca-cb8d6bccec4a","Type":"ContainerStarted","Data":"d9d62c11b7b5b25c5177b072b2a4321741de5f89d41ee15af98ede2c308a7605"}
Oct 30 19:09:03.666361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:03.666301    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podStartSLOduration=1.666283636 podStartE2EDuration="1.666283636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:03.663388052 +0000 UTC m=+1336.247871049" watchObservedRunningTime="2026-10-30 19:09:03.666283636 +0000 UTC m=+1336.250766633"
Oct 30 19:09:03.900979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:03.900933    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:09:03.915587 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:03.915551    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:09:04.042277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:04.042244    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:04.567718 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:04.567689    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:09:13.042877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:13.042840    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:13.055244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:13.055222    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:13.627588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:13.627559    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:09:16.186805 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:09:16.186763    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:09:18.689274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.689234    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"]
Oct 30 19:09:18.693174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.693153    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.791393 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.791359    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-330025\"/\"distinct-w2-330025-kserve-self-signed-certs\""
Oct 30 19:09:18.794888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.794859    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56e5bbf-8007-4b3a-af62-006001102f1e-tls-certs\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.795026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.794906    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-model-cache\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.795090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.795062    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-dshm\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.795138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.795095    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"]
Oct 30 19:09:18.795209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.795185    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-home\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.795268 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.795254    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-tmp-dir\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.795370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.795353    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvqm4\" (UniqueName: \"kubernetes.io/projected/f56e5bbf-8007-4b3a-af62-006001102f1e-kube-api-access-nvqm4\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.896543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.896498    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvqm4\" (UniqueName: \"kubernetes.io/projected/f56e5bbf-8007-4b3a-af62-006001102f1e-kube-api-access-nvqm4\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.896908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.896887    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56e5bbf-8007-4b3a-af62-006001102f1e-tls-certs\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.897065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.897051    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-model-cache\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.897180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.897168    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-dshm\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.897311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.897299    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-home\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.897434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.897421    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-tmp-dir\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.898016 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.897994    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-tmp-dir\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.899367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.899344    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-model-cache\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.899787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.899723    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-home\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.902650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.902608    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56e5bbf-8007-4b3a-af62-006001102f1e-tls-certs\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.903430 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.903410    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-dshm\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:18.982466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:18.982352    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvqm4\" (UniqueName: \"kubernetes.io/projected/f56e5bbf-8007-4b3a-af62-006001102f1e-kube-api-access-nvqm4\") pod \"distinct-w2-330025-kserve-55d67597b4-z4vcg\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") " pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:19.007834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:19.005995    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:19.483888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:19.483759    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"]
Oct 30 19:09:19.488179 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:09:19.488151    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf56e5bbf_8007_4b3a_af62_006001102f1e.slice/crio-db95e6800625c9dbacae228d300400c0bee37c074a9c2881008d1e1d5c6adede WatchSource:0}: Error finding container db95e6800625c9dbacae228d300400c0bee37c074a9c2881008d1e1d5c6adede: Status 404 returned error can't find the container with id db95e6800625c9dbacae228d300400c0bee37c074a9c2881008d1e1d5c6adede
Oct 30 19:09:19.654441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:19.654403    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" event={"ID":"f56e5bbf-8007-4b3a-af62-006001102f1e","Type":"ContainerStarted","Data":"c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b"}
Oct 30 19:09:19.654640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:19.654457    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" event={"ID":"f56e5bbf-8007-4b3a-af62-006001102f1e","Type":"ContainerStarted","Data":"db95e6800625c9dbacae228d300400c0bee37c074a9c2881008d1e1d5c6adede"}
Oct 30 19:09:19.769962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:19.769839    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podStartSLOduration=1.769806811 podStartE2EDuration="1.769806811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:19.767243935 +0000 UTC m=+1352.351726935" watchObservedRunningTime="2026-10-30 19:09:19.769806811 +0000 UTC m=+1352.354289807"
Oct 30 19:09:20.008461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:20.008428    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:29.008681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:29.008602    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:29.021365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:29.021343    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:29.728101 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:29.728070    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:09:44.413697 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.413657    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"]
Oct 30 19:09:44.418588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.418563    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435358    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m7xj\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-kube-api-access-6m7xj\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435412    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435571    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435626    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435742    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435771    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435822    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435959 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435856    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.435959 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.435887    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.469301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.469267    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-9275c3-openshift-default-dockercfg-hq6vr\""
Oct 30 19:09:44.540740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.540692    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.540991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.540778    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.540991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.540844    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.540991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.540872    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.540991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.540913    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.540991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.540952    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.541307 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.541004    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.541307 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.541039    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m7xj\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-kube-api-access-6m7xj\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.541307 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.541085    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.541948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.541879    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.542413 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.542389    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.542829 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.542789    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.543209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.543186    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.571173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.571133    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.571659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.571634    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.571769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.571695    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"]
Oct 30 19:09:44.573271 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.573244    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.646510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.646467    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.649599 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.649567    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m7xj\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-kube-api-access-6m7xj\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:44.733897 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:44.733782    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:45.188464 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:45.188424    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"]
Oct 30 19:09:45.225779 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:09:45.225748    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a151a48_f41c_4126_ae9c_db8f0d9874a0.slice/crio-8c38511dbdded2fd2a68e6fa2e18c190cf1ea3efed4df5dc12c1704f70f42719 WatchSource:0}: Error finding container 8c38511dbdded2fd2a68e6fa2e18c190cf1ea3efed4df5dc12c1704f70f42719: Status 404 returned error can't find the container with id 8c38511dbdded2fd2a68e6fa2e18c190cf1ea3efed4df5dc12c1704f70f42719
Oct 30 19:09:45.228775 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:45.228730    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:09:45.228891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:45.228855    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:09:45.228934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:45.228906    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:09:45.829086 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:45.829038    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" event={"ID":"0a151a48-f41c-4126-ae9c-db8f0d9874a0","Type":"ContainerStarted","Data":"63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e"}
Oct 30 19:09:45.829561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:45.829110    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" event={"ID":"0a151a48-f41c-4126-ae9c-db8f0d9874a0","Type":"ContainerStarted","Data":"8c38511dbdded2fd2a68e6fa2e18c190cf1ea3efed4df5dc12c1704f70f42719"}
Oct 30 19:09:45.951914 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:45.951852    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podStartSLOduration=1.951829577 podStartE2EDuration="1.951829577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:45.950801353 +0000 UTC m=+1378.535284351" watchObservedRunningTime="2026-10-30 19:09:45.951829577 +0000 UTC m=+1378.536312571"
Oct 30 19:09:46.190408 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:09:46.190372    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:09:46.734272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:46.734234    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:46.735700 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:46.735676    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:09:46.735808 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:46.735723    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 19:09:47.735106 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:47.735073    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:09:47.735478 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:47.735127    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 19:09:48.734751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:48.734715    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:09:48.735051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:48.735004    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 19:09:49.735171 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:49.735135    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:09:49.735592 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:49.735194    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 19:09:50.734579 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:50.734545    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:09:50.734874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:50.734601    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 19:09:51.734930 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:51.734892    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:09:51.735325 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:51.734958    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 19:09:52.738755 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:52.738717    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:52.739162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:52.739019    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:09:52.739675 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:09:52.739658    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:10:16.187552 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:10:16.187511    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:10:36.736968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.736928    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"]
Oct 30 19:10:36.740985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.740962    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:36.791485 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.791443    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:10:36.839096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.839065    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 19:10:36.842064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.842034    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"]
Oct 30 19:10:36.854406 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.854374    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:36.854612 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.854450    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plrlj\" (UniqueName: \"kubernetes.io/projected/40163aef-4889-4746-a224-e6d86aa36aa2-kube-api-access-plrlj\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:36.854612 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.854532    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:36.956203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.956166    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:36.956414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.956232    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plrlj\" (UniqueName: \"kubernetes.io/projected/40163aef-4889-4746-a224-e6d86aa36aa2-kube-api-access-plrlj\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:36.956414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.956275    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:36.957031 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.957005    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:36.958628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:36.958601    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:37.015727 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:37.015645    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plrlj\" (UniqueName: \"kubernetes.io/projected/40163aef-4889-4746-a224-e6d86aa36aa2-kube-api-access-plrlj\") pod \"maas-api-revoke-sub-060501da-8d4mc\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:37.055874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:37.055826    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:37.434517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:37.434481    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"]
Oct 30 19:10:37.438376 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:10:37.438345    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40163aef_4889_4746_a224_e6d86aa36aa2.slice/crio-0855f115841689b433ba18d0e41e8f7c3985585a1ee2409fd0058bfb95123707 WatchSource:0}: Error finding container 0855f115841689b433ba18d0e41e8f7c3985585a1ee2409fd0058bfb95123707: Status 404 returned error can't find the container with id 0855f115841689b433ba18d0e41e8f7c3985585a1ee2409fd0058bfb95123707
Oct 30 19:10:38.184932 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:38.184839    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc" event={"ID":"40163aef-4889-4746-a224-e6d86aa36aa2","Type":"ContainerStarted","Data":"f76c934e02260f9539e55e5a0f568fb486bb9d0c6a715873c0e14b503b137d85"}
Oct 30 19:10:38.184932 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:38.184895    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc" event={"ID":"40163aef-4889-4746-a224-e6d86aa36aa2","Type":"ContainerStarted","Data":"0855f115841689b433ba18d0e41e8f7c3985585a1ee2409fd0058bfb95123707"}
Oct 30 19:10:38.299107 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:38.299049    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc" podStartSLOduration=2.2990344670000002 podStartE2EDuration="2.299034467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:38.296216973 +0000 UTC m=+1430.880699970" watchObservedRunningTime="2026-10-30 19:10:38.299034467 +0000 UTC m=+1430.883517464"
Oct 30 19:10:39.192162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:39.192120    5232 generic.go:356] "Generic (PLEG): container finished" podID="40163aef-4889-4746-a224-e6d86aa36aa2" containerID="f76c934e02260f9539e55e5a0f568fb486bb9d0c6a715873c0e14b503b137d85" exitCode=0
Oct 30 19:10:39.192550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:39.192167    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc" event={"ID":"40163aef-4889-4746-a224-e6d86aa36aa2","Type":"ContainerDied","Data":"f76c934e02260f9539e55e5a0f568fb486bb9d0c6a715873c0e14b503b137d85"}
Oct 30 19:10:40.336891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.336865    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:40.494351 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.494273    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-xks-ca\") pod \"40163aef-4889-4746-a224-e6d86aa36aa2\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") "
Oct 30 19:10:40.494490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.494370    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40163aef-4889-4746-a224-e6d86aa36aa2-kube-api-access-plrlj\" (UniqueName: \"kubernetes.io/projected/40163aef-4889-4746-a224-e6d86aa36aa2-kube-api-access-plrlj\") pod \"40163aef-4889-4746-a224-e6d86aa36aa2\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") "
Oct 30 19:10:40.494490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.494393    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-openshift-ca\") pod \"40163aef-4889-4746-a224-e6d86aa36aa2\" (UID: \"40163aef-4889-4746-a224-e6d86aa36aa2\") "
Oct 30 19:10:40.494783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.494756    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-openshift-ca" pod "40163aef-4889-4746-a224-e6d86aa36aa2" (UID: "40163aef-4889-4746-a224-e6d86aa36aa2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:40.496301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.496278    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-xks-ca" pod "40163aef-4889-4746-a224-e6d86aa36aa2" (UID: "40163aef-4889-4746-a224-e6d86aa36aa2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:10:40.496392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.496357    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40163aef-4889-4746-a224-e6d86aa36aa2-kube-api-access-plrlj" pod "40163aef-4889-4746-a224-e6d86aa36aa2" (UID: "40163aef-4889-4746-a224-e6d86aa36aa2"). InnerVolumeSpecName "kube-api-access-plrlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:40.595743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.595709    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-xks-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:40.595743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.595737    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plrlj\" (UniqueName: \"kubernetes.io/projected/40163aef-4889-4746-a224-e6d86aa36aa2-kube-api-access-plrlj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:40.595743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:40.595747    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/40163aef-4889-4746-a224-e6d86aa36aa2-maas-api-openshift-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:41.205943 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:41.205911    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"
Oct 30 19:10:41.206139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:41.205936    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc" event={"ID":"40163aef-4889-4746-a224-e6d86aa36aa2","Type":"ContainerDied","Data":"0855f115841689b433ba18d0e41e8f7c3985585a1ee2409fd0058bfb95123707"}
Oct 30 19:10:41.206139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:41.206023    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0855f115841689b433ba18d0e41e8f7c3985585a1ee2409fd0058bfb95123707"
Oct 30 19:10:43.800229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.800190    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"]
Oct 30 19:10:43.801176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.800784    5232 cpu_manager.go:414] "containerMap: removing container" podUID="40163aef-4889-4746-a224-e6d86aa36aa2" containerName="revoke-keys"
Oct 30 19:10:43.801176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.800801    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40163aef-4889-4746-a224-e6d86aa36aa2" containerName="revoke-keys"
Oct 30 19:10:43.801176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.800956    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="40163aef-4889-4746-a224-e6d86aa36aa2" containerName="revoke-keys"
Oct 30 19:10:43.804523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.804503    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.905631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.905600    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"]
Oct 30 19:10:43.934730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.934695    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.934901 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.934737    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.934901 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.934881    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.934981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.934916    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.934981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.934934    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.935070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.935044    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.935134 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.935112    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.935172 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.935152    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k2sk\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-kube-api-access-9k2sk\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:43.935205 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:43.935176    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.036577 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.036536    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k2sk\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-kube-api-access-9k2sk\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.036773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.036593    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.036773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.036722    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.036773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.036752    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.036960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.036878    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.036960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.036925    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.036960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.036949    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.037114 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.036985    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.037114 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.037020    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.037223 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.037104    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.037223 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.037142    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.037335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.037317    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.037440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.037417    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.037829 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.037794    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.039393 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.039373    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.039637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.039616    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.096440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.096405    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.096602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.096578    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k2sk\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-kube-api-access-9k2sk\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.118424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.118394    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:44.548234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.548149    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"]
Oct 30 19:10:44.553559 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:10:44.553536    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbcebf7_74ce_4710_9dc3_a1bc64ba4229.slice/crio-83d0515866a6a416d38e54774ca1fd32fa73982049a51b1e1f68eb2a6caa4769 WatchSource:0}: Error finding container 83d0515866a6a416d38e54774ca1fd32fa73982049a51b1e1f68eb2a6caa4769: Status 404 returned error can't find the container with id 83d0515866a6a416d38e54774ca1fd32fa73982049a51b1e1f68eb2a6caa4769
Oct 30 19:10:44.555839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.555792    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:44.555925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.555880    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:44.555967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.555930    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:44.892771 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.892736    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"]
Oct 30 19:10:44.897070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.897046    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.947470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.947435    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.947655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.947501    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.947655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.947642    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.947787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.947712    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.947787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.947777    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.947963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.947905    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.947963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.947954    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v99wd\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-kube-api-access-v99wd\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.948115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.947996    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.948115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.948082    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:44.959765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:44.959730    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"]
Oct 30 19:10:45.049077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049035    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049101    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049152    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049209    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v99wd\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-kube-api-access-v99wd\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049254    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049298    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049380    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049484    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049489    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.049716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.049612    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.050237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.050192    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.050385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.050253    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.050385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.050289    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.050500 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.050477    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.051751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.051729    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.051874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.051771    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.109181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.109151    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.109365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.109339    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v99wd\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-kube-api-access-v99wd\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.210087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.210004    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:45.232992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.232954    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" event={"ID":"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229","Type":"ContainerStarted","Data":"69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac"}
Oct 30 19:10:45.233123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.233020    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" event={"ID":"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229","Type":"ContainerStarted","Data":"83d0515866a6a416d38e54774ca1fd32fa73982049a51b1e1f68eb2a6caa4769"}
Oct 30 19:10:45.666045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.665986    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podStartSLOduration=2.665969929 podStartE2EDuration="2.665969929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:45.367689684 +0000 UTC m=+1437.952172681" watchObservedRunningTime="2026-10-30 19:10:45.665969929 +0000 UTC m=+1438.250452926"
Oct 30 19:10:45.670834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.669398    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"]
Oct 30 19:10:45.675278 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:10:45.675248    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c34d0c_f7b9_45a9_8478_ca45c25dd632.slice/crio-0ca40d11116f0e26ed6b176493534b3581ac77dd4969f507cbb7086271d64e5a WatchSource:0}: Error finding container 0ca40d11116f0e26ed6b176493534b3581ac77dd4969f507cbb7086271d64e5a: Status 404 returned error can't find the container with id 0ca40d11116f0e26ed6b176493534b3581ac77dd4969f507cbb7086271d64e5a
Oct 30 19:10:45.678371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.678336    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:45.678466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.678405    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:45.678466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:45.678433    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:46.099727 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.099686    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"]
Oct 30 19:10:46.103872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.103841    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.118779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.118756    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:46.120509 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.120487    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:10:46.120607 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.120529    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 19:10:46.175362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.175329    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.175538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.175388    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af8fe62f-77ae-424a-94c6-54df1663d850-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.175590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.175554    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.175644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.175609    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.175704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.175659    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.175757 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.175703    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95nkb\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-kube-api-access-95nkb\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.175918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.175888    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.175971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.175936    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.176071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.176055    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af8fe62f-77ae-424a-94c6-54df1663d850-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.191939 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:10:46.191897    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:10:46.207949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.206604    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"]
Oct 30 19:10:46.266366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.266316    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" event={"ID":"28c34d0c-f7b9-45a9-8478-ca45c25dd632","Type":"ContainerStarted","Data":"668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020"}
Oct 30 19:10:46.266556 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.266392    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" event={"ID":"28c34d0c-f7b9-45a9-8478-ca45c25dd632","Type":"ContainerStarted","Data":"0ca40d11116f0e26ed6b176493534b3581ac77dd4969f507cbb7086271d64e5a"}
Oct 30 19:10:46.276940 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.276903    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.276998    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277058    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277117    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95nkb\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-kube-api-access-95nkb\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277184    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277215    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277286    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af8fe62f-77ae-424a-94c6-54df1663d850-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277444    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277577 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277524    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277705 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277529    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af8fe62f-77ae-424a-94c6-54df1663d850-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277710    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.277973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.277948    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.278092 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.278005    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.278315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.278297    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af8fe62f-77ae-424a-94c6-54df1663d850-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.279824 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.279792    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.279948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.279923    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af8fe62f-77ae-424a-94c6-54df1663d850-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.336201 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.336164    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.336379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.336299    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95nkb\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-kube-api-access-95nkb\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.387950 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.387835    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podStartSLOduration=2.387805251 podStartE2EDuration="2.387805251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:46.386038893 +0000 UTC m=+1438.970521903" watchObservedRunningTime="2026-10-30 19:10:46.387805251 +0000 UTC m=+1438.972288245"
Oct 30 19:10:46.418036 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.418003    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:46.919919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.919878    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"]
Oct 30 19:10:46.927730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.927688    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:46.927881 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.927761    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:46.927881 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:46.927793    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:47.118887 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.118850    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:10:47.119255 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.118913    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 19:10:47.212657 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.210889    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:47.212657 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.211859    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:10:47.212657 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.211910    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:10:47.295576 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.295535    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" event={"ID":"af8fe62f-77ae-424a-94c6-54df1663d850","Type":"ContainerStarted","Data":"e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6"}
Oct 30 19:10:47.295731 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.295595    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" event={"ID":"af8fe62f-77ae-424a-94c6-54df1663d850","Type":"ContainerStarted","Data":"5e874c1690b14d918bd0d2cacefb29958925266e8299ef22840e2087d82ab2ed"}
Oct 30 19:10:47.418571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.418532    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:47.426798 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:47.426741    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podStartSLOduration=1.426723052 podStartE2EDuration="1.426723052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:47.426448629 +0000 UTC m=+1440.010931626" watchObservedRunningTime="2026-10-30 19:10:47.426723052 +0000 UTC m=+1440.011206049"
Oct 30 19:10:48.119157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:48.119122    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:10:48.119649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:48.119173    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 19:10:48.213346 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:48.213275    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:10:48.213553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:48.213369    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:10:48.420016 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:48.419930    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:10:48.420016 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:48.419981    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 19:10:49.123837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:49.123184    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:10:49.123837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:49.123246    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 19:10:49.213832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:49.213177    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:10:49.213832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:49.213236    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:10:49.424425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:49.424327    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:10:49.424690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:49.424651    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 19:10:50.123175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.123139    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:10:50.123477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.123435    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 19:10:50.234839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.219596    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:10:50.234839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.219653    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:10:50.420892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.420212    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:10:50.420892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.420278    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 19:10:50.560002 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.559937    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:50.614718 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.614666    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:50.722889 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.718933    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:50.971212 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:50.971152    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:51.124838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.123307    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:10:51.124838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.123372    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 19:10:51.214839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.214464    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:10:51.214839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.214540    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:10:51.221832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.221054    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:51.337016 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.336958    5232 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:51.421434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.421223    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:10:51.421434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.421283    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 19:10:51.461728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.460916    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:51.562991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.562947    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:51.563637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.563615    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:10:51.579972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.579919    5232 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:51.616000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.615939    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:10:51.617841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.617718    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:51.619446 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.618606    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:10:51.815903 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.727139    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:51.815903 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:51.727561    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:10:52.092003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.091961    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:52.092609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.092582    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:10:52.171864 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.121321    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 19:10:52.172180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.172140    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 19:10:52.276862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.264892    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 19:10:52.276862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.265301    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:10:52.277692 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.277651    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:10:52.277942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.277901    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:10:52.327301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.327221    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:10:52.419187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.414119    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:10:52.419187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.418273    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:10:52.419187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.418320    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 19:10:52.420525 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.420484    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:10:52.430247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.430194    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:10:52.462926 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:52.440949    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:10:53.211320 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:53.211274    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:10:53.211502 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:53.211363    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:10:53.420628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:53.420593    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:10:53.421124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:53.420992    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 19:10:53.665869 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:53.662197    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:53.665869 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:53.663717    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:53.669517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:53.669487    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:10:54.419111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:54.419071    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:10:54.419368 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:54.419143    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 19:10:55.250985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:55.250943    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:10:55.251516 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:55.251015    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:55.725255 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:55.725212    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:55.725682 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:55.725652    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:55.726606 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:55.726588    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:10:55.821377 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:55.821340    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:55.821680 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:55.821659    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:10:55.822593 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:10:55.822569    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:11:13.545906 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.545827    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"]
Oct 30 19:11:13.550118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.550097    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.637623 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.637587    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"]
Oct 30 19:11:13.677646 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.677561    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.677646 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.677617    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.677902 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.677716    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.677902 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.677836    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.677902 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.677897    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe130c6-99a4-4c49-9e73-d364b9239099-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.678045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.677938    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.678189 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.678165    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.678256 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.678218    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmcsv\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-kube-api-access-fmcsv\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.678345 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.678325    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780044    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780095    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmcsv\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-kube-api-access-fmcsv\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780126    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780254    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780293    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780334    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780372    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780606 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780498    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780606 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780532    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe130c6-99a4-4c49-9e73-d364b9239099-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780606 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780572    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.780606 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.780594    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.781090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.781059    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.781242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.781173    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.781242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.781193    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe130c6-99a4-4c49-9e73-d364b9239099-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.782761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.782735    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.782906 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.782778    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.839702 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.839673    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.839909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.839806    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmcsv\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-kube-api-access-fmcsv\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:13.863619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:13.863584    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:14.291249 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:14.291147    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"]
Oct 30 19:11:14.295408 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:11:14.295382    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe130c6_99a4_4c49_9e73_d364b9239099.slice/crio-947caa0208969ff726fc143ea4ea2300bdfa9da2643f3b35570d2ac09b6a8e3a WatchSource:0}: Error finding container 947caa0208969ff726fc143ea4ea2300bdfa9da2643f3b35570d2ac09b6a8e3a: Status 404 returned error can't find the container with id 947caa0208969ff726fc143ea4ea2300bdfa9da2643f3b35570d2ac09b6a8e3a
Oct 30 19:11:14.297737 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:14.297704    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:14.297855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:14.297771    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:14.297855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:14.297802    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:14.953802 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:14.953753    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" event={"ID":"9fe130c6-99a4-4c49-9e73-d364b9239099","Type":"ContainerStarted","Data":"e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1"}
Oct 30 19:11:14.954226 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:14.953883    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" event={"ID":"9fe130c6-99a4-4c49-9e73-d364b9239099","Type":"ContainerStarted","Data":"947caa0208969ff726fc143ea4ea2300bdfa9da2643f3b35570d2ac09b6a8e3a"}
Oct 30 19:11:15.169401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:15.169333    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" podStartSLOduration=2.169319003 podStartE2EDuration="2.169319003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:15.119532824 +0000 UTC m=+1467.704015822" watchObservedRunningTime="2026-10-30 19:11:15.169319003 +0000 UTC m=+1467.753802000"
Oct 30 19:11:15.863900 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:15.863839    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:15.866101 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:15.866075    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:11:15.866256 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:15.866125    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 19:11:16.268223 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:11:16.268100    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:11:16.332186 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.330880    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"]
Oct 30 19:11:16.336194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.336169    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.423570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.423531    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xkmk\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-kube-api-access-7xkmk\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.423732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.423669    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.423732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.423714    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e46471e-6120-431a-bd06-666deece5f65-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.423806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.423796    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.423927 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.423907    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.424036 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.424019    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.424082 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.424068    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.424127 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.424111    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e46471e-6120-431a-bd06-666deece5f65-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.424171 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.424156    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.464833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.463930    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"]
Oct 30 19:11:16.531975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.531700    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.531975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.531773    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e46471e-6120-431a-bd06-666deece5f65-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.531975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.531918    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.532270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.531988    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.532270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.532022    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.532270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.532051    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.532270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.532090    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e46471e-6120-431a-bd06-666deece5f65-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.532270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.532131    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.532270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.532198    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xkmk\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-kube-api-access-7xkmk\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.570370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.533124    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.570370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.536569    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e46471e-6120-431a-bd06-666deece5f65-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.609837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.602270    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.609837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.602553    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.671838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.614449    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.671838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.616365    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.671838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.621642    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e46471e-6120-431a-bd06-666deece5f65-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.800554 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.800430    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.861836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.833006    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xkmk\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-kube-api-access-7xkmk\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:16.881963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.881928    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:11:16.882237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:16.882207    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 19:11:17.032987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:17.032950    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:17.876308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:17.876269    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:11:17.876947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:17.876912    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 19:11:18.424361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.424064    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:18.435151 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.434888    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 19:11:18.472365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.461630    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:18.472365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.463421    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:18.474786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.473733    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded"
Oct 30 19:11:18.717217 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.717171    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:18.718382 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.718350    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:18.718574 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.718550    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:18.795711 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.786172    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"]
Oct 30 19:11:18.865835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.865125    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:11:18.865835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:18.865176    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 19:11:19.032971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:19.032636    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" event={"ID":"5e46471e-6120-431a-bd06-666deece5f65","Type":"ContainerStarted","Data":"a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b"}
Oct 30 19:11:19.032971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:19.032704    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" event={"ID":"5e46471e-6120-431a-bd06-666deece5f65","Type":"ContainerStarted","Data":"b4db9d59255b48fc8ad363001a9795ce98a5a8ee49dd4b280f38de8bb523892c"}
Oct 30 19:11:19.034671 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:19.034621    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:19.050729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:19.048957    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:11:19.050729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:19.049025    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:11:19.158324 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:19.158259    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podStartSLOduration=3.158240495 podStartE2EDuration="3.158240495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:19.157172786 +0000 UTC m=+1471.741655787" watchObservedRunningTime="2026-10-30 19:11:19.158240495 +0000 UTC m=+1471.742723493"
Oct 30 19:11:19.866285 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:19.866247    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:11:19.866582 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:19.866542    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 19:11:20.034756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:20.034726    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:11:20.035378 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:20.035343    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:11:20.864191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:20.864153    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:11:20.864481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:20.864452    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 19:11:21.034732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:21.034697    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:11:21.034927 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:21.034750    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:11:22.034694 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:22.034651    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:11:22.034910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:22.034727    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:11:22.307141 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:22.307053    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:22.308096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:22.308073    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:22.308910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:22.308888    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:11:23.034510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:23.034472    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:11:23.034701 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:23.034528    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:11:24.035090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:24.035044    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:11:24.035485 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:24.035117    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:11:25.034905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:25.034868    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:11:25.035133 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:25.034947    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:11:26.185622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:26.185586    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:26.186595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:26.186560    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:26.189092 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:26.189068    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:11:31.326373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.326338    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"]
Oct 30 19:11:31.331402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.331378    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.396720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.396685    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.396945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.396752    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.396945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.396852    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0569546-94e1-4748-bcee-74965cf73a78-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.396945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.396889    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.397124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.396974    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.397124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.397027    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxzss\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-kube-api-access-kxzss\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.397124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.397089    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0569546-94e1-4748-bcee-74965cf73a78-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.397297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.397138    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.397297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.397251    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.433623 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.433591    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"]
Oct 30 19:11:31.484689 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.484659    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"]
Oct 30 19:11:31.489920 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.489895    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.498245 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498222    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.498374 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498300    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxzss\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-kube-api-access-kxzss\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.498440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498375    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0569546-94e1-4748-bcee-74965cf73a78-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.498440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498413    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.498542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498525    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.498605 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498583    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.498690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498669    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.498803 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498700    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0569546-94e1-4748-bcee-74965cf73a78-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.498803 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.498729    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.499211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.499183    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.499326 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.499233    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0569546-94e1-4748-bcee-74965cf73a78-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.499696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.499478    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.499696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.499629    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.499696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.499654    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.501077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.501056    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.502133 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.502113    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0569546-94e1-4748-bcee-74965cf73a78-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.549277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.549229    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"]
Oct 30 19:11:31.557431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.557404    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.557898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.557880    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxzss\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-kube-api-access-kxzss\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.600128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600094    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.600271 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600245    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.600324 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600293    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lds6n\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-kube-api-access-lds6n\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.600395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600372    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74805600-c429-41e2-8974-5c9c42079692-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.600433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600409    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.600549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600524    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74805600-c429-41e2-8974-5c9c42079692-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.600602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600587    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.600647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600631    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.600867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.600854    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.648905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.648850    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:31.702663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.702633    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lds6n\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-kube-api-access-lds6n\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.702790 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.702681    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74805600-c429-41e2-8974-5c9c42079692-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.702790 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.702714    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.702951 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.702792    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74805600-c429-41e2-8974-5c9c42079692-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.702951 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.702844    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.703116    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.703268    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.703464    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.703525    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74805600-c429-41e2-8974-5c9c42079692-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.703717    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.703790    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.703923    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704723 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.704427    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.704723 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.704590    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.706496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.706474    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.707022 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.707001    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74805600-c429-41e2-8974-5c9c42079692-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.763010 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.762979    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.763173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.763058    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lds6n\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-kube-api-access-lds6n\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:31.806441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:31.806407    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:32.096927 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.096892    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"]
Oct 30 19:11:32.101241 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:11:32.101214    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0569546_94e1_4748_bcee_74965cf73a78.slice/crio-9cf86b5af97fda614e431b268bde72e99c5c1dfbf23e80c114cdd0ffedc85155 WatchSource:0}: Error finding container 9cf86b5af97fda614e431b268bde72e99c5c1dfbf23e80c114cdd0ffedc85155: Status 404 returned error can't find the container with id 9cf86b5af97fda614e431b268bde72e99c5c1dfbf23e80c114cdd0ffedc85155
Oct 30 19:11:32.103499 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.103465    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.103559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.103532    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.103609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.103562    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.159978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.159949    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" event={"ID":"c0569546-94e1-4748-bcee-74965cf73a78","Type":"ContainerStarted","Data":"9cf86b5af97fda614e431b268bde72e99c5c1dfbf23e80c114cdd0ffedc85155"}
Oct 30 19:11:32.197912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.197841    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"]
Oct 30 19:11:32.200397 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:11:32.200371    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74805600_c429_41e2_8974_5c9c42079692.slice/crio-e8c7f761f729dfbf576ad35f70ac2de3a8ddb1310cfe666522b63f5660a0facb WatchSource:0}: Error finding container e8c7f761f729dfbf576ad35f70ac2de3a8ddb1310cfe666522b63f5660a0facb: Status 404 returned error can't find the container with id e8c7f761f729dfbf576ad35f70ac2de3a8ddb1310cfe666522b63f5660a0facb
Oct 30 19:11:32.203072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.203034    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.203179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.203107    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.203179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.203166    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.460667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.460590    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"]
Oct 30 19:11:32.464743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.464721    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.513401    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.513474    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlqv4\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-kube-api-access-qlqv4\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.513658    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.513696    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.513805    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.513855    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.513923    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.513952    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.518863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.514007    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.526379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.526352    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-942c5e85-openshift-default-dockercfg-2q6j6\""
Oct 30 19:11:32.535430 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.535398    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"]
Oct 30 19:11:32.637553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.618615    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.637855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.637804    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.638109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.638088    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.638236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.638218    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.638405 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.638389    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.638527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.638513    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.638706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.638690    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.638957 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.638938    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.641173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.639105    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlqv4\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-kube-api-access-qlqv4\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.649073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.649030    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.651805 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.651778    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.684839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.653696    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.684839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.654899    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.684839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.655121    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.684839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.655457    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.713944 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.708140    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.741239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.741170    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.769629 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.769328    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlqv4\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-kube-api-access-qlqv4\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:32.783269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:32.782638    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:33.192168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.192054    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" event={"ID":"74805600-c429-41e2-8974-5c9c42079692","Type":"ContainerStarted","Data":"883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b"}
Oct 30 19:11:33.192168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.192121    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" event={"ID":"74805600-c429-41e2-8974-5c9c42079692","Type":"ContainerStarted","Data":"e8c7f761f729dfbf576ad35f70ac2de3a8ddb1310cfe666522b63f5660a0facb"}
Oct 30 19:11:33.196388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.196213    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" event={"ID":"c0569546-94e1-4748-bcee-74965cf73a78","Type":"ContainerStarted","Data":"acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69"}
Oct 30 19:11:33.296008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.295957    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"]
Oct 30 19:11:33.300696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.300656    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:33.300791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.300731    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:33.300791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.300760    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:33.486258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.486204    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podStartSLOduration=2.486181634 podStartE2EDuration="2.486181634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:33.47241677 +0000 UTC m=+1486.056899771" watchObservedRunningTime="2026-10-30 19:11:33.486181634 +0000 UTC m=+1486.070664633"
Oct 30 19:11:33.487756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.487702    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podStartSLOduration=2.487688549 podStartE2EDuration="2.487688549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:33.334190049 +0000 UTC m=+1485.918673047" watchObservedRunningTime="2026-10-30 19:11:33.487688549 +0000 UTC m=+1486.072171547"
Oct 30 19:11:33.657127 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.657093    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:33.657427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.657407    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:33.657593 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.657571    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:33.822895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.819156    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:33.822895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.819264    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:33.822895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:33.819310    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:34.244838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.236905    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" event={"ID":"5fcbf443-86fd-4ea5-9c23-f6d7c957be18","Type":"ContainerStarted","Data":"06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104"}
Oct 30 19:11:34.244838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.236963    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" event={"ID":"5fcbf443-86fd-4ea5-9c23-f6d7c957be18","Type":"ContainerStarted","Data":"41e65387dfbf8894669071081c59d4190485e9ecbbb97d878df00f25282afb08"}
Oct 30 19:11:34.339852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.337927    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:34.474841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.427368    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:11:34.474841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.428141    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:34.474841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.450138    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:34.474841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.459335    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:34.658341 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.658296    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:34.658981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.658943    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:34.687991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.687942    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:34.791832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.785555    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:34.792737 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.792152    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:11:34.792737 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.792229    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:11:34.823060 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.819336    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:34.823060 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.819394    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:34.909890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:34.908899    5232 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.347980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.347912    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.349028 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.349001    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:11:35.450839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.446996    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.450839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.447802    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:11:35.495853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.495232    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded"
Oct 30 19:11:35.495853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.495614    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:11:35.516847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.498676    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podStartSLOduration=3.498658215 podStartE2EDuration="3.498658215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:34.427910843 +0000 UTC m=+1487.012393843" watchObservedRunningTime="2026-10-30 19:11:35.498658215 +0000 UTC m=+1488.083141214"
Oct 30 19:11:35.516847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.499577    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.525842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.517527    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:11:35.525842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.518436    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.525842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.523887    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 19:11:35.535852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.531257    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:11:35.652888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.651672    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:35.652888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.651728    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:35.756841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.690779    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.844838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.843356    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:11:35.874839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.849832    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:35.875152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:35.875114    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:36.162596 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.162563    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:36.429640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.350806    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:36.500161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.500011    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 19:11:36.501526 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.501498    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:36.503916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.503884    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:11:36.606203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.515269    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:36.681084 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.680996    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:11:36.684914 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.535272    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 19:11:36.703041 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.534000    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:36.703332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.703310    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:11:36.737836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.713141    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:36.738122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.738084    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:36.869182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.869142    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:36.888838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.869730    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:36.935055 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:36.934957    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:11:37.652841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.647915    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": context deadline exceeded"
Oct 30 19:11:37.652841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.648324    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 19:11:37.652841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.648597    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded"
Oct 30 19:11:37.692837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.691961    5232 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/health\": context deadline exceeded"
Oct 30 19:11:37.692837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.692074    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:11:37.717839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.694798    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:37.717839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.694863    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:37.786855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.747048    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:37.786855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.747371    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:37.926943 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.925918    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:37.926943 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.925974    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:37.959804 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:37.959756    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:38.672842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.653344    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.730916    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.731357    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": context deadline exceeded"
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.731643    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.747519    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.747832    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.808577    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.808631    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.811105    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:38.849054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.811151    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:38.960891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:38.960481    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:11:39.657969 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.657930    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:39.658254 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.658219    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:39.695035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.694973    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:39.733888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.731535    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": context deadline exceeded"
Oct 30 19:11:39.733888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.731926    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:11:39.759002 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.758957    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:39.759574 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.759539    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:39.759999 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.759974    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 19:11:39.809699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.809660    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:39.810035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.810003    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:39.923899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.923659    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:11:39.971855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:39.963949    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:40.015194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.015088    5232 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:40.568947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.568906    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:11:40.607573 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.569206    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:40.751855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.702389    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:40.751855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.702680    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:40.751855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.750489    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:40.751855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.732650    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:40.755833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.732688    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": context deadline exceeded"
Oct 30 19:11:40.778838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.759901    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:40.778838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.760207    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:40.814839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.810178    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:40.814839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.810233    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:40.914100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.914064    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:11:40.915375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.915349    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:11:40.922217 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.922194    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:11:40.927596 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:40.927575    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:11:41.104109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:41.104076    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:11:41.112583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:41.112556    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:11:41.675837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:41.673043    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:41.675837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:41.673107    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:41.809955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:41.808035    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:41.809955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:41.808100    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:42.662296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:42.662253    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:11:42.662916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:42.662881    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:11:42.807691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:42.807645    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:11:42.807901 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:42.807716    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 19:11:44.219644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:44.219611    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:44.220519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:44.220499    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:44.221182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:44.221163    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:11:44.272898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:44.272865    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:44.273585 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:44.273554    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:44.274190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:44.274159    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:11:46.188320 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:11:46.188281    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:11:47.733234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:47.733204    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:11:47.738316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:47.738293    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:11:47.744115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:47.744090    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:11:47.748138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:47.748114    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:11:58.962637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:58.962601    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"]
Oct 30 19:11:58.968003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:58.967980    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.057983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.057946    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"]
Oct 30 19:11:59.130460 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.130416    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx4jr\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-kube-api-access-dx4jr\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.130640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.130489    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/762a7039-237a-4de1-a8a6-af940f2581d3-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.130640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.130574    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/762a7039-237a-4de1-a8a6-af940f2581d3-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.130640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.130607    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.131232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.130638    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.131434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.131406    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.131563 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.131450    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.131563 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.131511    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.131701 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.131604    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.232719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.232630    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.232890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.232742    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.232890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.232799    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx4jr\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-kube-api-access-dx4jr\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.232890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.232849    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/762a7039-237a-4de1-a8a6-af940f2581d3-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.232925    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/762a7039-237a-4de1-a8a6-af940f2581d3-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.232951    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.232982    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.233010    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.233038    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.233345    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233439 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.233374    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233478 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.233436    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.233599    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/762a7039-237a-4de1-a8a6-af940f2581d3-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.233718 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.233697    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.235314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.235293    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.235436 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.235415    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/762a7039-237a-4de1-a8a6-af940f2581d3-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.293545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.293514    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx4jr\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-kube-api-access-dx4jr\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.293745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.293726    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:11:59.583853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:11:59.583739    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:12:00.018162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.018118    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"]
Oct 30 19:12:00.022550 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:12:00.022522    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762a7039_237a_4de1_a8a6_af940f2581d3.slice/crio-21c38535a29f62e0e2468772f7b51666e632a4b08af89e2f28c8f2bc341d28a8 WatchSource:0}: Error finding container 21c38535a29f62e0e2468772f7b51666e632a4b08af89e2f28c8f2bc341d28a8: Status 404 returned error can't find the container with id 21c38535a29f62e0e2468772f7b51666e632a4b08af89e2f28c8f2bc341d28a8
Oct 30 19:12:00.025057 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.025018    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:00.025152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.025108    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:00.025209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.025153    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:00.415386 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.415354    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" event={"ID":"762a7039-237a-4de1-a8a6-af940f2581d3","Type":"ContainerStarted","Data":"3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492"}
Oct 30 19:12:00.415561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.415405    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" event={"ID":"762a7039-237a-4de1-a8a6-af940f2581d3","Type":"ContainerStarted","Data":"21c38535a29f62e0e2468772f7b51666e632a4b08af89e2f28c8f2bc341d28a8"}
Oct 30 19:12:00.523007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.522976    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"]
Oct 30 19:12:00.527956 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.527931    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.594476 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.584373    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:12:00.594476 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.588929    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" podStartSLOduration=2.588914345 podStartE2EDuration="2.588914345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:00.587151044 +0000 UTC m=+1513.171634041" watchObservedRunningTime="2026-10-30 19:12:00.588914345 +0000 UTC m=+1513.173397335"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.661619    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.661736    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.661779    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.661829    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xtkz\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-kube-api-access-7xtkz\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.661870    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.661928    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.662063    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.662144    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.670918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.662191    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.673087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.673056    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"]
Oct 30 19:12:00.763075 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.763034    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.763295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.763188    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.763295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.763234    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.763420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.763334    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.763420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.763407    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.763543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.763455    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.763978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.763950    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.764094 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.764008    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xtkz\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-kube-api-access-7xtkz\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.764094 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.764049    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.764220 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.764093    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.764489 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.764465    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.764699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.764683    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.765189 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.765135    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.765781 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.765760    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.766544 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.766519    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.767297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.767278    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.861850 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.861376    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:00.864519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:00.864486    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xtkz\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-kube-api-access-7xtkz\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:01.149365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:01.148073    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:01.585506 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:01.585423    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:12:01.585506 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:01.585477    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:12:01.609084 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:01.608929    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"]
Oct 30 19:12:01.612362 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:12:01.612334    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc2f14b_8fb0_4435_97eb_72440e29cdb9.slice/crio-6aeabb6662175ece39feb55d50ff9dd04b690b51f5cdd680de70f3ac5c323156 WatchSource:0}: Error finding container 6aeabb6662175ece39feb55d50ff9dd04b690b51f5cdd680de70f3ac5c323156: Status 404 returned error can't find the container with id 6aeabb6662175ece39feb55d50ff9dd04b690b51f5cdd680de70f3ac5c323156
Oct 30 19:12:01.615153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:01.615120    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:01.615227 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:01.615187    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:01.615227 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:01.615223    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:02.432524 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:02.432493    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" event={"ID":"8dc2f14b-8fb0-4435-97eb-72440e29cdb9","Type":"ContainerStarted","Data":"d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505"}
Oct 30 19:12:02.432956 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:02.432542    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" event={"ID":"8dc2f14b-8fb0-4435-97eb-72440e29cdb9","Type":"ContainerStarted","Data":"6aeabb6662175ece39feb55d50ff9dd04b690b51f5cdd680de70f3ac5c323156"}
Oct 30 19:12:02.590840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:02.586202    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:12:02.590840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:02.586263    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:12:02.635969 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:02.635916    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podStartSLOduration=2.6358914970000002 podStartE2EDuration="2.635891497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:02.60252764 +0000 UTC m=+1515.187010637" watchObservedRunningTime="2026-10-30 19:12:02.635891497 +0000 UTC m=+1515.220374494"
Oct 30 19:12:03.160839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:03.160729    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:03.161068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:03.161000    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:12:03.161141 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:03.161058    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 19:12:03.586898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:03.586334    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:12:03.586898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:03.586394    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:12:04.158837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:04.158256    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:12:04.158837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:04.158323    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 19:12:04.589006 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:04.588967    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:12:04.589993 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:04.589959    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:12:05.168837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:05.165252    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:12:05.168837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:05.165321    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 19:12:05.588832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:05.588135    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:12:05.588832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:05.588452    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:12:06.158183 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:06.158143    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:12:06.158862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:06.158750    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 19:12:06.585049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:06.584956    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:12:06.585049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:06.585026    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:12:07.161224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:07.161184    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:12:07.161844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:07.161784    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 19:12:07.817690 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:07.817657    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:12:07.818129 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:07.818101    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:12:07.818932 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:07.818914    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:12:08.156185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:08.156145    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:12:08.156358 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:08.156211    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 19:12:08.916312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:08.916271    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"]
Oct 30 19:12:08.919828 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:08.919787    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:08.970835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:08.970491    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-942c5e85-serving-cert\""
Oct 30 19:12:09.022653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.022614    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"]
Oct 30 19:12:09.047370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.047329    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ghbv\" (UniqueName: \"kubernetes.io/projected/f8d74b9a-50fb-429c-a202-c1ae9e782834-kube-api-access-4ghbv\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.047546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.047455    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d74b9a-50fb-429c-a202-c1ae9e782834-openshift-service-ca\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.047652 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.047607    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.047709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.047694    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-metrics-certs\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.149226 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.149172    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ghbv\" (UniqueName: \"kubernetes.io/projected/f8d74b9a-50fb-429c-a202-c1ae9e782834-kube-api-access-4ghbv\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.149424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.149245    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d74b9a-50fb-429c-a202-c1ae9e782834-openshift-service-ca\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.149424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.149383    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.149542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.149437    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-metrics-certs\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.149764 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:12:09.149726    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-serving-cert: secret "maas-api-e2e-mt-942c5e85-serving-cert" not found
Oct 30 19:12:09.150050 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:12:09.150031    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls podName:f8d74b9a-50fb-429c-a202-c1ae9e782834 nodeName:}" failed. No retries permitted until 2026-10-30 19:12:09.649988653 +0000 UTC m=+1522.234471647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls") pod "maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt" (UID: "f8d74b9a-50fb-429c-a202-c1ae9e782834") : secret "maas-api-e2e-mt-942c5e85-serving-cert" not found
Oct 30 19:12:09.150704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.150678    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d74b9a-50fb-429c-a202-c1ae9e782834-openshift-service-ca\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.153860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.152974    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-metrics-certs\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.221777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.220390    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ghbv\" (UniqueName: \"kubernetes.io/projected/f8d74b9a-50fb-429c-a202-c1ae9e782834-kube-api-access-4ghbv\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.433613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.433577    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:09.434106 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.434073    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:09.435142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.435124    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:12:09.673239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.673199    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.675643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.675612    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:09.835572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:09.835541    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:10.252456 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:10.250213    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"]
Oct 30 19:12:10.260791 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:12:10.260759    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d74b9a_50fb_429c_a202_c1ae9e782834.slice/crio-1a5edddead674875f4bc2b5b67fa82fa832ddde8ca6f9ab3c169d34f8ecdbcce WatchSource:0}: Error finding container 1a5edddead674875f4bc2b5b67fa82fa832ddde8ca6f9ab3c169d34f8ecdbcce: Status 404 returned error can't find the container with id 1a5edddead674875f4bc2b5b67fa82fa832ddde8ca6f9ab3c169d34f8ecdbcce
Oct 30 19:12:10.262554 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:10.262527    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:12:10.545065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:10.544975    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt" event={"ID":"f8d74b9a-50fb-429c-a202-c1ae9e782834","Type":"ContainerStarted","Data":"1a5edddead674875f4bc2b5b67fa82fa832ddde8ca6f9ab3c169d34f8ecdbcce"}
Oct 30 19:12:11.553139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:11.553050    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt" event={"ID":"f8d74b9a-50fb-429c-a202-c1ae9e782834","Type":"ContainerStarted","Data":"c94add83c5d153a5bf23d8665b714992fd9f36b95960fb989ba83ca02d2a66d1"}
Oct 30 19:12:11.553516 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:11.553178    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:11.676613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:11.676401    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt" podStartSLOduration=2.706506638 podStartE2EDuration="3.676383985s" totalImagesPullingTime="969.877347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:11.672541114 +0000 UTC m=+1524.257024112" watchObservedRunningTime="2026-10-30 19:12:11.676383985 +0000 UTC m=+1524.260866983"
Oct 30 19:12:13.637192 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:13.632908    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:13.683967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:13.683911    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded"
Oct 30 19:12:13.792838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:13.792470    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:13.969890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:13.968899    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:14.053836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.052953    5232 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:14.118836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.117945    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 19:12:14.509841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.507919    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": context deadline exceeded"
Oct 30 19:12:14.640835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.640485    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:14.644835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.642490    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:12:14.687704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.685900    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:14.687704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.686308    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:12:14.984951 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.984901    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:14.985959 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:14.985935    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:12:15.111847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:15.095297    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:12:15.433353 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:15.433321    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:12:15.437294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:15.437269    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:12:16.266839 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:12:16.263228    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:12:17.685071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:17.685033    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:12:29.212855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.212770    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"]
Oct 30 19:12:29.217293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.217275    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364205 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364167    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364308    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364403    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364552 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364503    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj25m\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-kube-api-access-rj25m\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364694    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364771 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364755    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364831 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364798    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364915 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364897    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.364971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.364957    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.428998 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.428944    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"]
Oct 30 19:12:29.466459 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466370    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466459 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466421    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466459 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466444    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466747 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466490    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466747 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466550    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466747 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466629    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466747 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466736    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466986 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466858    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466986 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466894    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj25m\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-kube-api-access-rj25m\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466986 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466897    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.466986 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.466906    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.467586 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.467554    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.467586 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.467574    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.467783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.467592    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.469261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.469232    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.469600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.469583    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.526872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.526838    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.527736 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.527717    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj25m\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-kube-api-access-rj25m\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.531667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.531650    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:29.969013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.968985    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"]
Oct 30 19:12:29.972038 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:12:29.972014    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d5ea4c_0972_4f5a_9cbe_dbf9da2b7729.slice/crio-6476de0dd1d1e080bc9160cecbc8351c207f72a6a34b0b93521ce418b65085c3 WatchSource:0}: Error finding container 6476de0dd1d1e080bc9160cecbc8351c207f72a6a34b0b93521ce418b65085c3: Status 404 returned error can't find the container with id 6476de0dd1d1e080bc9160cecbc8351c207f72a6a34b0b93521ce418b65085c3
Oct 30 19:12:29.974363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.974326    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:29.974442 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.974396    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:29.974490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:29.974441    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:30.078130 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:30.078018    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" event={"ID":"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729","Type":"ContainerStarted","Data":"abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b"}
Oct 30 19:12:30.078130 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:30.078090    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" event={"ID":"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729","Type":"ContainerStarted","Data":"6476de0dd1d1e080bc9160cecbc8351c207f72a6a34b0b93521ce418b65085c3"}
Oct 30 19:12:30.197788 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:30.197738    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" podStartSLOduration=1.197725709 podStartE2EDuration="1.197725709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:30.195893392 +0000 UTC m=+1542.780376391" watchObservedRunningTime="2026-10-30 19:12:30.197725709 +0000 UTC m=+1542.782208706"
Oct 30 19:12:30.531984 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:30.531953    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:31.533160 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:31.533128    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:12:31.533535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:31.533189    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:12:32.532870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:32.532835    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:12:32.533043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:32.532892    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:12:33.532213 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:33.532179    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:12:33.532610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:33.532231    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:12:34.532135 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:34.532096    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:12:34.532383 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:34.532161    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:12:35.532058 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:35.532019    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:12:35.532263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:35.532078    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:12:36.532646 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:36.532609    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:12:36.533091 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:36.532665    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:12:37.541898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:37.541870    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:37.542342 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:37.542150    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:37.543008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:37.542990    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:12:38.241988 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.241957    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"]
Oct 30 19:12:38.246080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.246051    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.296775 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.296745    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 19:12:38.296987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.296747    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 19:12:38.296987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.296886    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 19:12:38.297072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.297017    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4r545\""
Oct 30 19:12:38.303838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.303796    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.303989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.303865    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.304060 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.303988    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.304060 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.304033    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/90484b62-de48-4b7a-8e3b-da5c11f9862d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.304190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.304169    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.304232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.304210    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.304330 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.304316    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmgbc\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-kube-api-access-nmgbc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.349807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.349774    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"]
Oct 30 19:12:38.405839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.405784    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.405983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.405853    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.405983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.405932    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmgbc\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-kube-api-access-nmgbc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.405983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.405978    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.406150 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.406011    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.406150 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.406046    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.406150 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.406069    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/90484b62-de48-4b7a-8e3b-da5c11f9862d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.406844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.406797    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.408289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.408266    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.408420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.408350    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.408420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.408325    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/90484b62-de48-4b7a-8e3b-da5c11f9862d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.408588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.408567    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.464982 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.464948    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.465133 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.465115    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmgbc\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-kube-api-access-nmgbc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nqjqb\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:38.561041 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:38.560956    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:39.006841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:39.006796    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"]
Oct 30 19:12:39.010483 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:12:39.010455    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90484b62_de48_4b7a_8e3b_da5c11f9862d.slice/crio-be14e4ee2237e72afac1a93d56096ca0d1ff9eead011e16a07f0a2df58a18eac WatchSource:0}: Error finding container be14e4ee2237e72afac1a93d56096ca0d1ff9eead011e16a07f0a2df58a18eac: Status 404 returned error can't find the container with id be14e4ee2237e72afac1a93d56096ca0d1ff9eead011e16a07f0a2df58a18eac
Oct 30 19:12:39.145302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:39.145261    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb" event={"ID":"90484b62-de48-4b7a-8e3b-da5c11f9862d","Type":"ContainerStarted","Data":"be14e4ee2237e72afac1a93d56096ca0d1ff9eead011e16a07f0a2df58a18eac"}
Oct 30 19:12:43.270095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:43.269958    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:43.270095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:43.270021    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:44.184752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:44.184713    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb" event={"ID":"90484b62-de48-4b7a-8e3b-da5c11f9862d","Type":"ContainerStarted","Data":"0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9"}
Oct 30 19:12:44.184950 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:44.184808    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:44.186736 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:44.186704    5232 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-nqjqb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:12:44.186895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:44.186771    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb" podUID="90484b62-de48-4b7a-8e3b-da5c11f9862d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:12:44.398838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:44.394771    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb" podStartSLOduration=2.138543434 podStartE2EDuration="6.394748593s" totalImagesPullingTime="4.256205159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:44.353841916 +0000 UTC m=+1556.938324961" watchObservedRunningTime="2026-10-30 19:12:44.394748593 +0000 UTC m=+1556.979231590"
Oct 30 19:12:45.192738 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:45.192708    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:12:46.330007 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:12:46.329788    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:12:47.748960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:47.748894    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:48.106837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:48.106017    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:48.116365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:48.116317    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:48.441199 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:48.441096    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:48.442054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:48.442024    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": context deadline exceeded"
Oct 30 19:12:48.512395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:48.511289    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:48.512395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:48.511761    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:48.762891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:48.757348    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:48.762891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:48.758157    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:12:49.143971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.143923    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:49.292878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.256299    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:12:49.292878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.145302    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:49.292878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.276633    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:12:49.466885 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.462912    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:49.466885 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.463683    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:12:49.517009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.516947    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:49.526833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.517659    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 19:12:49.527509 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.527483    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:12:49.533832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.527692    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:12:49.786925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.759973    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:49.942490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.942459    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 19:12:49.986354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.986325    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 19:12:49.994707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:49.993141    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:12:50.006774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:50.005284    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:12:50.006774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:50.006211    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:12:50.006774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:50.006273    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:12:57.479141 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:57.479104    5232 generic.go:356] "Generic (PLEG): container finished" podID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerID="06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104" exitCode=0
Oct 30 19:12:57.479558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:57.479183    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" event={"ID":"5fcbf443-86fd-4ea5-9c23-f6d7c957be18","Type":"ContainerDied","Data":"06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104"}
Oct 30 19:12:57.479649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:57.479632    5232 scope.go:122] "RemoveContainer" containerID="06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104"
Oct 30 19:12:57.481200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:57.481169    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:57.481336 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:57.481263    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:57.481336 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:57.481314    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:58.487715 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:58.487682    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" event={"ID":"5fcbf443-86fd-4ea5-9c23-f6d7c957be18","Type":"ContainerStarted","Data":"d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12"}
Oct 30 19:12:58.784892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:58.783726    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:12:58.785570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:58.785156    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:12:58.785570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:58.785212    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:12:59.783083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:59.783049    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:12:59.783518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:59.783104    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:12:59.964673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:59.964624    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"]
Oct 30 19:12:59.968335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:12:59.968313    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.009556 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.009518    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs4gq\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-kube-api-access-cs4gq\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.009761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.009590    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.009761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.009669    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.009761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.009718    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.009761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.009744    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.009999 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.009789    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.009999 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.009945    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.010106 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.010009    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.010106 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.010065    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.074202 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.074121    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"]
Oct 30 19:13:00.111088 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111056    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111284 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111098    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111284 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111223    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111284 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111258    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111289    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111390    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111474    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111519    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111724 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111654    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs4gq\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-kube-api-access-cs4gq\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.111968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111939    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.112123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.111949    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.112317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.112288    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.112700 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.112669    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.113104 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.113062    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.114209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.114184    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.119662 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.119640    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.173286 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.173250    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.176078 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.176049    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs4gq\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-kube-api-access-cs4gq\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.280588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.280560    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:00.776177 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.776080    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"]
Oct 30 19:13:00.785470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.785437    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:13:00.785957 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.785502    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:13:00.803045 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:13:00.803017    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d23bf28_1715_4699_b69f_ff91ff6dcfb4.slice/crio-71ec67347fdfb628e9784b51eb64edb17edd04ff67e6f9228c33098adb73510c WatchSource:0}: Error finding container 71ec67347fdfb628e9784b51eb64edb17edd04ff67e6f9228c33098adb73510c: Status 404 returned error can't find the container with id 71ec67347fdfb628e9784b51eb64edb17edd04ff67e6f9228c33098adb73510c
Oct 30 19:13:00.806017 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.805976    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:00.806096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.806073    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:00.806138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:00.806118    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:01.517312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:01.517277    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" event={"ID":"3d23bf28-1715-4699-b69f-ff91ff6dcfb4","Type":"ContainerStarted","Data":"8e6b163f60eecb9211cae81f66b17ce081a414d507005b9b50536ff01126952f"}
Oct 30 19:13:01.517483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:01.517334    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" event={"ID":"3d23bf28-1715-4699-b69f-ff91ff6dcfb4","Type":"ContainerStarted","Data":"71ec67347fdfb628e9784b51eb64edb17edd04ff67e6f9228c33098adb73510c"}
Oct 30 19:13:01.636424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:01.636369    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podStartSLOduration=2.6363550829999998 podStartE2EDuration="2.636355083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:01.633952348 +0000 UTC m=+1574.218435345" watchObservedRunningTime="2026-10-30 19:13:01.636355083 +0000 UTC m=+1574.220838080"
Oct 30 19:13:01.786373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:01.784482    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:13:01.786373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:01.784560    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:13:02.304836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:02.291512    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:02.304836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:02.291690    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:13:02.304836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:02.291770    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:13:02.810834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:02.796286    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:13:02.810834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:02.796379    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:13:03.283838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:03.281302    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:13:03.283838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:03.281388    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:13:03.786838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:03.786238    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:13:03.786838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:03.786322    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:13:04.281245 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:04.281204    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:13:04.281877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:04.281836    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:13:04.787642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:04.787596    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:13:04.788083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:04.788052    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:13:04.788924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:04.788904    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:13:05.281420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:05.281380    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:13:05.281868 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:05.281452    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:13:06.281049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:06.281006    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:13:06.281232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:06.281079    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:13:07.281981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:07.281939    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:13:07.282479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:07.282010    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:13:08.281083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:08.281045    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:13:08.281280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:08.281104    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:13:10.186575 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:10.186535    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:10.187149 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:10.187063    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:10.187968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:13:10.187944    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:13:16.189547 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:13:16.189502    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:13:46.195119 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:13:46.195052    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:14:16.189484 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:14:16.189441    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:14:27.276211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.276155    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"]
Oct 30 19:14:27.290999 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.289691    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.385835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.366713    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"]
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.414726    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-certs\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.414796    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-podinfo\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.414852    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-socket\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.414921    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-token\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.415043    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-data\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.415105    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkx6v\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-kube-api-access-qkx6v\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.415150    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-credential-socket\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.415190    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-envoy\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.422955 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.415317    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istiod-ca-cert\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.447489 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.447234    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0d2ef6-openshift-default-dockercfg-82xmw\""
Oct 30 19:14:27.517071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517031    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-socket\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517115    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-token\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517214    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-data\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517410 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517286    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkx6v\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-kube-api-access-qkx6v\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517410 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517332    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-credential-socket\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517529 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517510    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-envoy\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517565    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-data\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517662 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517643    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istiod-ca-cert\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517697    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-certs\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517718    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-credential-socket\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.517773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.517735    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-podinfo\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.518023 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.518000    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-socket\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.518280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.518259    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-certs\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.518625 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.518605    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istiod-ca-cert\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.522072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.520727    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-envoy\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.522615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.522592    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-podinfo\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.579153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.579072    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-token\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.579904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.579860    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkx6v\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-kube-api-access-qkx6v\") pod \"e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") " pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:27.640411 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:27.640372    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:28.093393 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:28.093359    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"]
Oct 30 19:14:28.096881 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:14:28.096850    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff4f45b_6dbb_41e5_9a75_59aa00b8712e.slice/crio-71f0f62f1bc8385ad1702d185a553fc91f091327c8f281381b0aa225c2626845 WatchSource:0}: Error finding container 71f0f62f1bc8385ad1702d185a553fc91f091327c8f281381b0aa225c2626845: Status 404 returned error can't find the container with id 71f0f62f1bc8385ad1702d185a553fc91f091327c8f281381b0aa225c2626845
Oct 30 19:14:28.099344 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:28.099307    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:14:28.099453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:28.099391    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:14:28.099453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:28.099434    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:14:28.464733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:28.464632    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" event={"ID":"cff4f45b-6dbb-41e5-9a75-59aa00b8712e","Type":"ContainerStarted","Data":"fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3"}
Oct 30 19:14:28.464733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:28.464699    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" event={"ID":"cff4f45b-6dbb-41e5-9a75-59aa00b8712e","Type":"ContainerStarted","Data":"71f0f62f1bc8385ad1702d185a553fc91f091327c8f281381b0aa225c2626845"}
Oct 30 19:14:28.596657 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:28.596602    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" podStartSLOduration=1.596587177 podStartE2EDuration="1.596587177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:28.59557691 +0000 UTC m=+1661.180059910" watchObservedRunningTime="2026-10-30 19:14:28.596587177 +0000 UTC m=+1661.181070173"
Oct 30 19:14:28.641384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:28.641358    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:29.646989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:29.646951    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:30.480005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:30.479972    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:30.480908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:30.480877    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:14:46.189034 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:14:46.188992    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:14:48.401784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.401749    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"]
Oct 30 19:14:48.406244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.406223    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.431334 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.431308    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcbrz\" (UniqueName: \"kubernetes.io/projected/5011494a-45e2-49d1-8a53-404a1b54f7d4-kube-api-access-zcbrz\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.431551 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.431526    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-metrics-certs\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.431656 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.431642    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.431751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.431733    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5011494a-45e2-49d1-8a53-404a1b54f7d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.457902 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.457860    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f44636-serving-cert\""
Oct 30 19:14:48.509373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.509331    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"]
Oct 30 19:14:48.532715 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.532676    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5011494a-45e2-49d1-8a53-404a1b54f7d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.532952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.532780    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcbrz\" (UniqueName: \"kubernetes.io/projected/5011494a-45e2-49d1-8a53-404a1b54f7d4-kube-api-access-zcbrz\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.533062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.533028    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-metrics-certs\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.533156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.533139    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.533352 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:14:48.533334    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-serving-cert: secret "maas-api-e2e-api-a-f44636-serving-cert" not found
Oct 30 19:14:48.533432 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:14:48.533408    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls podName:5011494a-45e2-49d1-8a53-404a1b54f7d4 nodeName:}" failed. No retries permitted until 2026-10-30 19:14:49.033388743 +0000 UTC m=+1681.617871739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls") pod "maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7" (UID: "5011494a-45e2-49d1-8a53-404a1b54f7d4") : secret "maas-api-e2e-api-a-f44636-serving-cert" not found
Oct 30 19:14:48.533530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.533430    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5011494a-45e2-49d1-8a53-404a1b54f7d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.535968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.535943    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-metrics-certs\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:48.594277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:48.594246    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcbrz\" (UniqueName: \"kubernetes.io/projected/5011494a-45e2-49d1-8a53-404a1b54f7d4-kube-api-access-zcbrz\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:49.039780 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:49.039736    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:49.042150 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:49.042127    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls\") pod \"maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:49.321535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:49.321455    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:49.703142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:49.703109    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"]
Oct 30 19:14:49.706677 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:14:49.706648    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5011494a_45e2_49d1_8a53_404a1b54f7d4.slice/crio-bd64d2e49fb3b1db6891fe46a69b1b2a6554d313846858468668ec467d6fa558 WatchSource:0}: Error finding container bd64d2e49fb3b1db6891fe46a69b1b2a6554d313846858468668ec467d6fa558: Status 404 returned error can't find the container with id bd64d2e49fb3b1db6891fe46a69b1b2a6554d313846858468668ec467d6fa558
Oct 30 19:14:50.635034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:50.634991    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7" event={"ID":"5011494a-45e2-49d1-8a53-404a1b54f7d4","Type":"ContainerStarted","Data":"bd64d2e49fb3b1db6891fe46a69b1b2a6554d313846858468668ec467d6fa558"}
Oct 30 19:14:51.643598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:51.643562    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7" event={"ID":"5011494a-45e2-49d1-8a53-404a1b54f7d4","Type":"ContainerStarted","Data":"043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2"}
Oct 30 19:14:51.644067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:51.643646    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:14:51.761601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:51.761538    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7" podStartSLOduration=2.906795487 podStartE2EDuration="3.761520613s" totalImagesPullingTime="854.725126ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:51.758957255 +0000 UTC m=+1684.343440253" watchObservedRunningTime="2026-10-30 19:14:51.761520613 +0000 UTC m=+1684.346003615"
Oct 30 19:14:57.656144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:14:57.656116    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:15:00.178233 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.178198    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"]
Oct 30 19:15:00.182864 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.182839    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:00.208570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.208542    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"]
Oct 30 19:15:00.212909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.212887    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:00.215376 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.215352    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"]
Oct 30 19:15:00.219377 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.219361    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:00.231488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.231464    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"]
Oct 30 19:15:00.235791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.235772    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:00.364838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.352951    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 19:15:00.389847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.354797    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p85pg\" (UniqueName: \"kubernetes.io/projected/c25f605a-89a5-4b5b-b3ed-348bd91e541c-kube-api-access-p85pg\") pod \"maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx\" (UID: \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:00.389847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.386021    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c25f605a-89a5-4b5b-b3ed-348bd91e541c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx\" (UID: \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:00.389847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.386076    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbrcw\" (UniqueName: \"kubernetes.io/projected/25cd8f03-12af-4ebd-b751-278123bb4390-kube-api-access-jbrcw\") pod \"maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74\" (UID: \"25cd8f03-12af-4ebd-b751-278123bb4390\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:00.389847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.386137    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3b7657-e14f-4f73-b80a-ea8215c51157-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m\" (UID: \"2c3b7657-e14f-4f73-b80a-ea8215c51157\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:00.389847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.386237    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss97p\" (UniqueName: \"kubernetes.io/projected/2c3b7657-e14f-4f73-b80a-ea8215c51157-kube-api-access-ss97p\") pod \"maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m\" (UID: \"2c3b7657-e14f-4f73-b80a-ea8215c51157\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:00.389847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.386297    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtzgw\" (UniqueName: \"kubernetes.io/projected/1f3533ee-bc99-445c-9d65-62a1f00b3a85-kube-api-access-xtzgw\") pod \"maas-api-key-cleanup-29889795-thgdm\" (UID: \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:00.389847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.386565    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3533ee-bc99-445c-9d65-62a1f00b3a85-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889795-thgdm\" (UID: \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:00.389847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.386598    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd8f03-12af-4ebd-b751-278123bb4390-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74\" (UID: \"25cd8f03-12af-4ebd-b751-278123bb4390\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:00.425542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.425509    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"]
Oct 30 19:15:00.436356 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.436117    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"]
Oct 30 19:15:00.455678 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.455617    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"]
Oct 30 19:15:00.491167 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.491107    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"]
Oct 30 19:15:00.493227 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.493198    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss97p\" (UniqueName: \"kubernetes.io/projected/2c3b7657-e14f-4f73-b80a-ea8215c51157-kube-api-access-ss97p\") pod \"maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m\" (UID: \"2c3b7657-e14f-4f73-b80a-ea8215c51157\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:00.500535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.500498    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtzgw\" (UniqueName: \"kubernetes.io/projected/1f3533ee-bc99-445c-9d65-62a1f00b3a85-kube-api-access-xtzgw\") pod \"maas-api-key-cleanup-29889795-thgdm\" (UID: \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:00.500943 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.500920    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3533ee-bc99-445c-9d65-62a1f00b3a85-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889795-thgdm\" (UID: \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:00.502154 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.502129    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd8f03-12af-4ebd-b751-278123bb4390-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74\" (UID: \"25cd8f03-12af-4ebd-b751-278123bb4390\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:00.503115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.503089    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p85pg\" (UniqueName: \"kubernetes.io/projected/c25f605a-89a5-4b5b-b3ed-348bd91e541c-kube-api-access-p85pg\") pod \"maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx\" (UID: \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:00.504110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.504090    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c25f605a-89a5-4b5b-b3ed-348bd91e541c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx\" (UID: \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:00.504246 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.504228    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbrcw\" (UniqueName: \"kubernetes.io/projected/25cd8f03-12af-4ebd-b751-278123bb4390-kube-api-access-jbrcw\") pod \"maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74\" (UID: \"25cd8f03-12af-4ebd-b751-278123bb4390\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:00.504386 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.504373    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3b7657-e14f-4f73-b80a-ea8215c51157-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m\" (UID: \"2c3b7657-e14f-4f73-b80a-ea8215c51157\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:00.505733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.505711    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd8f03-12af-4ebd-b751-278123bb4390-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74\" (UID: \"25cd8f03-12af-4ebd-b751-278123bb4390\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:00.506867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.506653    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3533ee-bc99-445c-9d65-62a1f00b3a85-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889795-thgdm\" (UID: \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:00.506996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.506974    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3b7657-e14f-4f73-b80a-ea8215c51157-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m\" (UID: \"2c3b7657-e14f-4f73-b80a-ea8215c51157\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:00.507131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.507110    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c25f605a-89a5-4b5b-b3ed-348bd91e541c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx\" (UID: \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:00.603950 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.598566    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss97p\" (UniqueName: \"kubernetes.io/projected/2c3b7657-e14f-4f73-b80a-ea8215c51157-kube-api-access-ss97p\") pod \"maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m\" (UID: \"2c3b7657-e14f-4f73-b80a-ea8215c51157\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:00.655587 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.655552    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtzgw\" (UniqueName: \"kubernetes.io/projected/1f3533ee-bc99-445c-9d65-62a1f00b3a85-kube-api-access-xtzgw\") pod \"maas-api-key-cleanup-29889795-thgdm\" (UID: \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:00.710602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.710515    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p85pg\" (UniqueName: \"kubernetes.io/projected/c25f605a-89a5-4b5b-b3ed-348bd91e541c-kube-api-access-p85pg\") pod \"maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx\" (UID: \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:00.712916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.712887    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbrcw\" (UniqueName: \"kubernetes.io/projected/25cd8f03-12af-4ebd-b751-278123bb4390-kube-api-access-jbrcw\") pod \"maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74\" (UID: \"25cd8f03-12af-4ebd-b751-278123bb4390\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:00.797542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.797502    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:00.833203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.833170    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:00.840482 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:00.840455    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:01.006996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.006837    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:01.266997 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.265842    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"]
Oct 30 19:15:01.312825 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.311717    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"]
Oct 30 19:15:01.320366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.320053    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"]
Oct 30 19:15:01.323078 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:15:01.323051    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3b7657_e14f_4f73_b80a_ea8215c51157.slice/crio-2c867181584f719baf0250a4627ae3bb802add1186286778528209e6e849566e WatchSource:0}: Error finding container 2c867181584f719baf0250a4627ae3bb802add1186286778528209e6e849566e: Status 404 returned error can't find the container with id 2c867181584f719baf0250a4627ae3bb802add1186286778528209e6e849566e
Oct 30 19:15:01.342961 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:15:01.342926    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3533ee_bc99_445c_9d65_62a1f00b3a85.slice/crio-d9a1c7875b8b73ac50c7a025d77e23366362e313e07c08831d182f9e4489bb48 WatchSource:0}: Error finding container d9a1c7875b8b73ac50c7a025d77e23366362e313e07c08831d182f9e4489bb48: Status 404 returned error can't find the container with id d9a1c7875b8b73ac50c7a025d77e23366362e313e07c08831d182f9e4489bb48
Oct 30 19:15:01.597220 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.597176    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"]
Oct 30 19:15:01.740620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.740583    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" event={"ID":"c25f605a-89a5-4b5b-b3ed-348bd91e541c","Type":"ContainerStarted","Data":"3a598fd1b5d5ddadab5b3fd314baf6f161b1d05e2ece183dc6967bb11628af1c"}
Oct 30 19:15:01.742583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.742538    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" event={"ID":"25cd8f03-12af-4ebd-b751-278123bb4390","Type":"ContainerStarted","Data":"66b9227f6a5d97a9deb78eed7676b5aafe225317456b51f55a61ead0080e24c2"}
Oct 30 19:15:01.742921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.742895    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" event={"ID":"25cd8f03-12af-4ebd-b751-278123bb4390","Type":"ContainerStarted","Data":"b56ca6ecbf3bb6f5ffb5d5a92df9fbe5f7427971338734d8784f19671418fe8d"}
Oct 30 19:15:01.744094 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.744070    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" event={"ID":"1f3533ee-bc99-445c-9d65-62a1f00b3a85","Type":"ContainerStarted","Data":"d9a1c7875b8b73ac50c7a025d77e23366362e313e07c08831d182f9e4489bb48"}
Oct 30 19:15:01.745293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.745274    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" event={"ID":"2c3b7657-e14f-4f73-b80a-ea8215c51157","Type":"ContainerStarted","Data":"2c867181584f719baf0250a4627ae3bb802add1186286778528209e6e849566e"}
Oct 30 19:15:01.856480 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:01.856430    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" podStartSLOduration=1.856416675 podStartE2EDuration="1.856416675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:01.854923265 +0000 UTC m=+1694.439406351" watchObservedRunningTime="2026-10-30 19:15:01.856416675 +0000 UTC m=+1694.440899704"
Oct 30 19:15:02.754179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.754090    5232 generic.go:356] "Generic (PLEG): container finished" podID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerID="7d2c6cb8ce73a150242978e069a1d2531f44ed580cba0fa347cb91341e4c2f1d" exitCode=60
Oct 30 19:15:02.754633 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.754171    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" event={"ID":"1f3533ee-bc99-445c-9d65-62a1f00b3a85","Type":"ContainerDied","Data":"7d2c6cb8ce73a150242978e069a1d2531f44ed580cba0fa347cb91341e4c2f1d"}
Oct 30 19:15:02.754633 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.754547    5232 scope.go:122] "RemoveContainer" containerID="7d2c6cb8ce73a150242978e069a1d2531f44ed580cba0fa347cb91341e4c2f1d"
Oct 30 19:15:02.756043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.756017    5232 generic.go:356] "Generic (PLEG): container finished" podID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerID="4ddc50fb5516030e2e4802ae2ee864bfc2d084d150e4dc11f0cd4ea7489186cd" exitCode=60
Oct 30 19:15:02.756132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.756103    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" event={"ID":"2c3b7657-e14f-4f73-b80a-ea8215c51157","Type":"ContainerDied","Data":"4ddc50fb5516030e2e4802ae2ee864bfc2d084d150e4dc11f0cd4ea7489186cd"}
Oct 30 19:15:02.756350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.756336    5232 scope.go:122] "RemoveContainer" containerID="4ddc50fb5516030e2e4802ae2ee864bfc2d084d150e4dc11f0cd4ea7489186cd"
Oct 30 19:15:02.758133 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.758092    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" event={"ID":"c25f605a-89a5-4b5b-b3ed-348bd91e541c","Type":"ContainerStarted","Data":"92f7cd736e9cb10c95c74296a6cf3fd4ec7e035b0dc5aec92fff92c41ea63b51"}
Oct 30 19:15:02.759953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.759931    5232 generic.go:356] "Generic (PLEG): container finished" podID="25cd8f03-12af-4ebd-b751-278123bb4390" containerID="66b9227f6a5d97a9deb78eed7676b5aafe225317456b51f55a61ead0080e24c2" exitCode=60
Oct 30 19:15:02.760049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.760010    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" event={"ID":"25cd8f03-12af-4ebd-b751-278123bb4390","Type":"ContainerDied","Data":"66b9227f6a5d97a9deb78eed7676b5aafe225317456b51f55a61ead0080e24c2"}
Oct 30 19:15:02.760193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:02.760179    5232 scope.go:122] "RemoveContainer" containerID="66b9227f6a5d97a9deb78eed7676b5aafe225317456b51f55a61ead0080e24c2"
Oct 30 19:15:03.219057 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.218995    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" podStartSLOduration=3.218979993 podStartE2EDuration="3.218979993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:03.213610981 +0000 UTC m=+1695.798093977" watchObservedRunningTime="2026-10-30 19:15:03.218979993 +0000 UTC m=+1695.803462990"
Oct 30 19:15:03.770206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.770171    5232 generic.go:356] "Generic (PLEG): container finished" podID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerID="175a743f6e775e3cf8a436d11bf8984f0d599d6fafc8f57d77953bcd1f8ccd5c" exitCode=60
Oct 30 19:15:03.770703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.770257    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" event={"ID":"1f3533ee-bc99-445c-9d65-62a1f00b3a85","Type":"ContainerDied","Data":"175a743f6e775e3cf8a436d11bf8984f0d599d6fafc8f57d77953bcd1f8ccd5c"}
Oct 30 19:15:03.770703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.770318    5232 scope.go:122] "RemoveContainer" containerID="7d2c6cb8ce73a150242978e069a1d2531f44ed580cba0fa347cb91341e4c2f1d"
Oct 30 19:15:03.770703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.770617    5232 scope.go:122] "RemoveContainer" containerID="175a743f6e775e3cf8a436d11bf8984f0d599d6fafc8f57d77953bcd1f8ccd5c"
Oct 30 19:15:03.771050 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:03.771028    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889795-thgdm_odh-ai-gateway-infra(1f3533ee-bc99-445c-9d65-62a1f00b3a85)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85"
Oct 30 19:15:03.772516 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.772493    5232 generic.go:356] "Generic (PLEG): container finished" podID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerID="ce87e2ed53cd59b4bbfa30cf725b8a46796a987aff30cd836461e37c6d9e0c66" exitCode=60
Oct 30 19:15:03.772625 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.772580    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" event={"ID":"2c3b7657-e14f-4f73-b80a-ea8215c51157","Type":"ContainerDied","Data":"ce87e2ed53cd59b4bbfa30cf725b8a46796a987aff30cd836461e37c6d9e0c66"}
Oct 30 19:15:03.772985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.772963    5232 scope.go:122] "RemoveContainer" containerID="ce87e2ed53cd59b4bbfa30cf725b8a46796a987aff30cd836461e37c6d9e0c66"
Oct 30 19:15:03.773277 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:03.773255    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m_odh-ai-gateway-infra(2c3b7657-e14f-4f73-b80a-ea8215c51157)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157"
Oct 30 19:15:03.774210 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.774190    5232 generic.go:356] "Generic (PLEG): container finished" podID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerID="92f7cd736e9cb10c95c74296a6cf3fd4ec7e035b0dc5aec92fff92c41ea63b51" exitCode=60
Oct 30 19:15:03.774300 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.774265    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" event={"ID":"c25f605a-89a5-4b5b-b3ed-348bd91e541c","Type":"ContainerDied","Data":"92f7cd736e9cb10c95c74296a6cf3fd4ec7e035b0dc5aec92fff92c41ea63b51"}
Oct 30 19:15:03.774573 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.774553    5232 scope.go:122] "RemoveContainer" containerID="92f7cd736e9cb10c95c74296a6cf3fd4ec7e035b0dc5aec92fff92c41ea63b51"
Oct 30 19:15:03.776317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.776295    5232 generic.go:356] "Generic (PLEG): container finished" podID="25cd8f03-12af-4ebd-b751-278123bb4390" containerID="0ed4f2c2ee7b29d0278ca3fa6f8e858c486f1201aaf112257778c3a4669cbcb4" exitCode=60
Oct 30 19:15:03.776406 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.776390    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" event={"ID":"25cd8f03-12af-4ebd-b751-278123bb4390","Type":"ContainerDied","Data":"0ed4f2c2ee7b29d0278ca3fa6f8e858c486f1201aaf112257778c3a4669cbcb4"}
Oct 30 19:15:03.776649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.776634    5232 scope.go:122] "RemoveContainer" containerID="0ed4f2c2ee7b29d0278ca3fa6f8e858c486f1201aaf112257778c3a4669cbcb4"
Oct 30 19:15:03.777514 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:03.777491    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74_odh-ai-gateway-infra(25cd8f03-12af-4ebd-b751-278123bb4390)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" podUID="25cd8f03-12af-4ebd-b751-278123bb4390"
Oct 30 19:15:03.792732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.792705    5232 scope.go:122] "RemoveContainer" containerID="4ddc50fb5516030e2e4802ae2ee864bfc2d084d150e4dc11f0cd4ea7489186cd"
Oct 30 19:15:03.813275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:03.813253    5232 scope.go:122] "RemoveContainer" containerID="66b9227f6a5d97a9deb78eed7676b5aafe225317456b51f55a61ead0080e24c2"
Oct 30 19:15:04.785963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:04.785927    5232 generic.go:356] "Generic (PLEG): container finished" podID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerID="d360673e1a367f9903bca393290fc4ebd0c2d3fcf8bca335825d449bbf8ab462" exitCode=60
Oct 30 19:15:04.786510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:04.786010    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" event={"ID":"c25f605a-89a5-4b5b-b3ed-348bd91e541c","Type":"ContainerDied","Data":"d360673e1a367f9903bca393290fc4ebd0c2d3fcf8bca335825d449bbf8ab462"}
Oct 30 19:15:04.786510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:04.786081    5232 scope.go:122] "RemoveContainer" containerID="92f7cd736e9cb10c95c74296a6cf3fd4ec7e035b0dc5aec92fff92c41ea63b51"
Oct 30 19:15:04.786510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:04.786442    5232 scope.go:122] "RemoveContainer" containerID="d360673e1a367f9903bca393290fc4ebd0c2d3fcf8bca335825d449bbf8ab462"
Oct 30 19:15:04.786845 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:04.786786    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx_odh-ai-gateway-infra(c25f605a-89a5-4b5b-b3ed-348bd91e541c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c"
Oct 30 19:15:04.788241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:04.788222    5232 scope.go:122] "RemoveContainer" containerID="0ed4f2c2ee7b29d0278ca3fa6f8e858c486f1201aaf112257778c3a4669cbcb4"
Oct 30 19:15:04.788608 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:04.788582    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74_odh-ai-gateway-infra(25cd8f03-12af-4ebd-b751-278123bb4390)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" podUID="25cd8f03-12af-4ebd-b751-278123bb4390"
Oct 30 19:15:04.789965 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:04.789948    5232 scope.go:122] "RemoveContainer" containerID="175a743f6e775e3cf8a436d11bf8984f0d599d6fafc8f57d77953bcd1f8ccd5c"
Oct 30 19:15:04.790277 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:04.790253    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889795-thgdm_odh-ai-gateway-infra(1f3533ee-bc99-445c-9d65-62a1f00b3a85)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85"
Oct 30 19:15:04.791546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:04.791528    5232 scope.go:122] "RemoveContainer" containerID="ce87e2ed53cd59b4bbfa30cf725b8a46796a987aff30cd836461e37c6d9e0c66"
Oct 30 19:15:04.791780 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:04.791760    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m_odh-ai-gateway-infra(2c3b7657-e14f-4f73-b80a-ea8215c51157)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157"
Oct 30 19:15:05.801282 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:05.801256    5232 scope.go:122] "RemoveContainer" containerID="d360673e1a367f9903bca393290fc4ebd0c2d3fcf8bca335825d449bbf8ab462"
Oct 30 19:15:05.801676 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:05.801581    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx_odh-ai-gateway-infra(c25f605a-89a5-4b5b-b3ed-348bd91e541c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c"
Oct 30 19:15:15.533137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:15.533107    5232 scope.go:122] "RemoveContainer" containerID="ce87e2ed53cd59b4bbfa30cf725b8a46796a987aff30cd836461e37c6d9e0c66"
Oct 30 19:15:15.533505 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:15.533201    5232 scope.go:122] "RemoveContainer" containerID="175a743f6e775e3cf8a436d11bf8984f0d599d6fafc8f57d77953bcd1f8ccd5c"
Oct 30 19:15:15.875572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:15.875536    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" event={"ID":"1f3533ee-bc99-445c-9d65-62a1f00b3a85","Type":"ContainerStarted","Data":"de515dd1b1641936705590b684e24a5ca7a4ee3bce594e5816402f44c6923ec3"}
Oct 30 19:15:15.877666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:15.877643    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" event={"ID":"2c3b7657-e14f-4f73-b80a-ea8215c51157","Type":"ContainerStarted","Data":"b1f5fea6df794e11a3b76f7586719a6fea55880bfce63e55934a8ab6eeaff28f"}
Oct 30 19:15:15.991114 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:15.991059    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" podStartSLOduration=15.991041695 podStartE2EDuration="15.991041695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:15.987642763 +0000 UTC m=+1708.572125760" watchObservedRunningTime="2026-10-30 19:15:15.991041695 +0000 UTC m=+1708.575524692"
Oct 30 19:15:16.103182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.103075    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" podStartSLOduration=16.103059056 podStartE2EDuration="16.103059056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:16.100006299 +0000 UTC m=+1708.684489297" watchObservedRunningTime="2026-10-30 19:15:16.103059056 +0000 UTC m=+1708.687542059"
Oct 30 19:15:16.190526 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:16.190488    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:15:16.888720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.888680    5232 generic.go:356] "Generic (PLEG): container finished" podID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerID="de515dd1b1641936705590b684e24a5ca7a4ee3bce594e5816402f44c6923ec3" exitCode=60
Oct 30 19:15:16.889232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.888765    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" event={"ID":"1f3533ee-bc99-445c-9d65-62a1f00b3a85","Type":"ContainerDied","Data":"de515dd1b1641936705590b684e24a5ca7a4ee3bce594e5816402f44c6923ec3"}
Oct 30 19:15:16.889232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.888910    5232 scope.go:122] "RemoveContainer" containerID="175a743f6e775e3cf8a436d11bf8984f0d599d6fafc8f57d77953bcd1f8ccd5c"
Oct 30 19:15:16.889367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.889350    5232 scope.go:122] "RemoveContainer" containerID="de515dd1b1641936705590b684e24a5ca7a4ee3bce594e5816402f44c6923ec3"
Oct 30 19:15:16.889760 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:16.889736    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889795-thgdm_odh-ai-gateway-infra(1f3533ee-bc99-445c-9d65-62a1f00b3a85)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85"
Oct 30 19:15:16.891388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.891367    5232 generic.go:356] "Generic (PLEG): container finished" podID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerID="b1f5fea6df794e11a3b76f7586719a6fea55880bfce63e55934a8ab6eeaff28f" exitCode=60
Oct 30 19:15:16.891467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.891438    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" event={"ID":"2c3b7657-e14f-4f73-b80a-ea8215c51157","Type":"ContainerDied","Data":"b1f5fea6df794e11a3b76f7586719a6fea55880bfce63e55934a8ab6eeaff28f"}
Oct 30 19:15:16.892013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.891991    5232 scope.go:122] "RemoveContainer" containerID="b1f5fea6df794e11a3b76f7586719a6fea55880bfce63e55934a8ab6eeaff28f"
Oct 30 19:15:16.892468 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:16.892446    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m_odh-ai-gateway-infra(2c3b7657-e14f-4f73-b80a-ea8215c51157)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157"
Oct 30 19:15:16.905311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:16.905282    5232 scope.go:122] "RemoveContainer" containerID="ce87e2ed53cd59b4bbfa30cf725b8a46796a987aff30cd836461e37c6d9e0c66"
Oct 30 19:15:17.002251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:17.002215    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"]
Oct 30 19:15:17.113573 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:17.113536    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"]
Oct 30 19:15:17.539934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:17.539905    5232 scope.go:122] "RemoveContainer" containerID="d360673e1a367f9903bca393290fc4ebd0c2d3fcf8bca335825d449bbf8ab462"
Oct 30 19:15:17.900217 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:17.900179    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" event={"ID":"c25f605a-89a5-4b5b-b3ed-348bd91e541c","Type":"ContainerStarted","Data":"af3cded7112ea55f04c136ff595e21ecde9907dcc80f917a9b18d297530da2da"}
Oct 30 19:15:18.100729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.100673    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:18.129485 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.129463    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:18.168997 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.168963    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f3533ee-bc99-445c-9d65-62a1f00b3a85-kube-api-access-xtzgw\" (UniqueName: \"kubernetes.io/projected/1f3533ee-bc99-445c-9d65-62a1f00b3a85-kube-api-access-xtzgw\") pod \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\" (UID: \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\") "
Oct 30 19:15:18.169190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.169065    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c3b7657-e14f-4f73-b80a-ea8215c51157-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3b7657-e14f-4f73-b80a-ea8215c51157-maas-api-service-ca\") pod \"2c3b7657-e14f-4f73-b80a-ea8215c51157\" (UID: \"2c3b7657-e14f-4f73-b80a-ea8215c51157\") "
Oct 30 19:15:18.169190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.169114    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c3b7657-e14f-4f73-b80a-ea8215c51157-kube-api-access-ss97p\" (UniqueName: \"kubernetes.io/projected/2c3b7657-e14f-4f73-b80a-ea8215c51157-kube-api-access-ss97p\") pod \"2c3b7657-e14f-4f73-b80a-ea8215c51157\" (UID: \"2c3b7657-e14f-4f73-b80a-ea8215c51157\") "
Oct 30 19:15:18.169190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.169140    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f3533ee-bc99-445c-9d65-62a1f00b3a85-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3533ee-bc99-445c-9d65-62a1f00b3a85-maas-api-service-ca\") pod \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\" (UID: \"1f3533ee-bc99-445c-9d65-62a1f00b3a85\") "
Oct 30 19:15:18.169545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.169520    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c3b7657-e14f-4f73-b80a-ea8215c51157-maas-api-service-ca" pod "2c3b7657-e14f-4f73-b80a-ea8215c51157" (UID: "2c3b7657-e14f-4f73-b80a-ea8215c51157"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:18.169545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.169524    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f3533ee-bc99-445c-9d65-62a1f00b3a85-maas-api-service-ca" pod "1f3533ee-bc99-445c-9d65-62a1f00b3a85" (UID: "1f3533ee-bc99-445c-9d65-62a1f00b3a85"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:18.169681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.169663    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3b7657-e14f-4f73-b80a-ea8215c51157-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.169717 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.169685    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3533ee-bc99-445c-9d65-62a1f00b3a85-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.172149 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.172122    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f3533ee-bc99-445c-9d65-62a1f00b3a85-kube-api-access-xtzgw" pod "1f3533ee-bc99-445c-9d65-62a1f00b3a85" (UID: "1f3533ee-bc99-445c-9d65-62a1f00b3a85"). InnerVolumeSpecName "kube-api-access-xtzgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:18.172256 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.172183    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c3b7657-e14f-4f73-b80a-ea8215c51157-kube-api-access-ss97p" pod "2c3b7657-e14f-4f73-b80a-ea8215c51157" (UID: "2c3b7657-e14f-4f73-b80a-ea8215c51157"). InnerVolumeSpecName "kube-api-access-ss97p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:18.270753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.270720    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss97p\" (UniqueName: \"kubernetes.io/projected/2c3b7657-e14f-4f73-b80a-ea8215c51157-kube-api-access-ss97p\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.270753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.270751    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtzgw\" (UniqueName: \"kubernetes.io/projected/1f3533ee-bc99-445c-9d65-62a1f00b3a85-kube-api-access-xtzgw\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.533630 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.533597    5232 scope.go:122] "RemoveContainer" containerID="0ed4f2c2ee7b29d0278ca3fa6f8e858c486f1201aaf112257778c3a4669cbcb4"
Oct 30 19:15:18.919012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.915707    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"
Oct 30 19:15:18.919012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.915734    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm" event={"ID":"1f3533ee-bc99-445c-9d65-62a1f00b3a85","Type":"ContainerDied","Data":"d9a1c7875b8b73ac50c7a025d77e23366362e313e07c08831d182f9e4489bb48"}
Oct 30 19:15:18.919012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.915878    5232 scope.go:122] "RemoveContainer" containerID="de515dd1b1641936705590b684e24a5ca7a4ee3bce594e5816402f44c6923ec3"
Oct 30 19:15:18.919012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.917372    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"
Oct 30 19:15:18.919012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.917471    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m" event={"ID":"2c3b7657-e14f-4f73-b80a-ea8215c51157","Type":"ContainerDied","Data":"2c867181584f719baf0250a4627ae3bb802add1186286778528209e6e849566e"}
Oct 30 19:15:18.924861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.920178    5232 generic.go:356] "Generic (PLEG): container finished" podID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerID="af3cded7112ea55f04c136ff595e21ecde9907dcc80f917a9b18d297530da2da" exitCode=60
Oct 30 19:15:18.924861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.920254    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" event={"ID":"c25f605a-89a5-4b5b-b3ed-348bd91e541c","Type":"ContainerDied","Data":"af3cded7112ea55f04c136ff595e21ecde9907dcc80f917a9b18d297530da2da"}
Oct 30 19:15:18.924861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.920688    5232 scope.go:122] "RemoveContainer" containerID="af3cded7112ea55f04c136ff595e21ecde9907dcc80f917a9b18d297530da2da"
Oct 30 19:15:18.924861 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:18.921032    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx_odh-ai-gateway-infra(c25f605a-89a5-4b5b-b3ed-348bd91e541c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c"
Oct 30 19:15:18.929242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.929214    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" event={"ID":"25cd8f03-12af-4ebd-b751-278123bb4390","Type":"ContainerStarted","Data":"ebf32d4f891375d19537766ac9335d9f900b54ec0dd37575feef41069dbb3278"}
Oct 30 19:15:18.930392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.930375    5232 scope.go:122] "RemoveContainer" containerID="b1f5fea6df794e11a3b76f7586719a6fea55880bfce63e55934a8ab6eeaff28f"
Oct 30 19:15:18.939625 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:18.939605    5232 scope.go:122] "RemoveContainer" containerID="d360673e1a367f9903bca393290fc4ebd0c2d3fcf8bca335825d449bbf8ab462"
Oct 30 19:15:19.028108 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.028018    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"]
Oct 30 19:15:19.188599 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.188560    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"]
Oct 30 19:15:19.192444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.192415    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-thgdm"]
Oct 30 19:15:19.297311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.297224    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"]
Oct 30 19:15:19.301909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.301880    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f44636-29889795-jbj8m"]
Oct 30 19:15:19.540605 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.540574    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" path="/var/lib/kubelet/pods/1f3533ee-bc99-445c-9d65-62a1f00b3a85/volumes"
Oct 30 19:15:19.541036 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.541022    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" path="/var/lib/kubelet/pods/2c3b7657-e14f-4f73-b80a-ea8215c51157/volumes"
Oct 30 19:15:19.945796 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.945755    5232 generic.go:356] "Generic (PLEG): container finished" podID="25cd8f03-12af-4ebd-b751-278123bb4390" containerID="ebf32d4f891375d19537766ac9335d9f900b54ec0dd37575feef41069dbb3278" exitCode=60
Oct 30 19:15:19.946235 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.945848    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" event={"ID":"25cd8f03-12af-4ebd-b751-278123bb4390","Type":"ContainerDied","Data":"ebf32d4f891375d19537766ac9335d9f900b54ec0dd37575feef41069dbb3278"}
Oct 30 19:15:19.946235 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.945910    5232 scope.go:122] "RemoveContainer" containerID="0ed4f2c2ee7b29d0278ca3fa6f8e858c486f1201aaf112257778c3a4669cbcb4"
Oct 30 19:15:19.946400 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:19.946319    5232 scope.go:122] "RemoveContainer" containerID="ebf32d4f891375d19537766ac9335d9f900b54ec0dd37575feef41069dbb3278"
Oct 30 19:15:19.946705 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:19.946682    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74_odh-ai-gateway-infra(25cd8f03-12af-4ebd-b751-278123bb4390)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" podUID="25cd8f03-12af-4ebd-b751-278123bb4390"
Oct 30 19:15:20.125751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.125722    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:20.200982 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.200896    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c25f605a-89a5-4b5b-b3ed-348bd91e541c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c25f605a-89a5-4b5b-b3ed-348bd91e541c-maas-api-service-ca\") pod \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\" (UID: \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\") "
Oct 30 19:15:20.201142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.201008    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c25f605a-89a5-4b5b-b3ed-348bd91e541c-kube-api-access-p85pg\" (UniqueName: \"kubernetes.io/projected/c25f605a-89a5-4b5b-b3ed-348bd91e541c-kube-api-access-p85pg\") pod \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\" (UID: \"c25f605a-89a5-4b5b-b3ed-348bd91e541c\") "
Oct 30 19:15:20.201263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.201241    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c25f605a-89a5-4b5b-b3ed-348bd91e541c-maas-api-service-ca" pod "c25f605a-89a5-4b5b-b3ed-348bd91e541c" (UID: "c25f605a-89a5-4b5b-b3ed-348bd91e541c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:20.201507 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.201491    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c25f605a-89a5-4b5b-b3ed-348bd91e541c-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:20.203063 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.203044    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c25f605a-89a5-4b5b-b3ed-348bd91e541c-kube-api-access-p85pg" pod "c25f605a-89a5-4b5b-b3ed-348bd91e541c" (UID: "c25f605a-89a5-4b5b-b3ed-348bd91e541c"). InnerVolumeSpecName "kube-api-access-p85pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:20.303170 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.303136    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p85pg\" (UniqueName: \"kubernetes.io/projected/c25f605a-89a5-4b5b-b3ed-348bd91e541c-kube-api-access-p85pg\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:20.426503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.426465    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"]
Oct 30 19:15:20.957307 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.957259    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"
Oct 30 19:15:20.958068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.957258    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx" event={"ID":"c25f605a-89a5-4b5b-b3ed-348bd91e541c","Type":"ContainerDied","Data":"3a598fd1b5d5ddadab5b3fd314baf6f161b1d05e2ece183dc6967bb11628af1c"}
Oct 30 19:15:20.958068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:20.957411    5232 scope.go:122] "RemoveContainer" containerID="af3cded7112ea55f04c136ff595e21ecde9907dcc80f917a9b18d297530da2da"
Oct 30 19:15:21.117579 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.117547    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"]
Oct 30 19:15:21.124046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.124016    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4f4614-29889795-jjtrx"]
Oct 30 19:15:21.165907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.165882    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:21.216305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.216226    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25cd8f03-12af-4ebd-b751-278123bb4390-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd8f03-12af-4ebd-b751-278123bb4390-maas-api-service-ca\") pod \"25cd8f03-12af-4ebd-b751-278123bb4390\" (UID: \"25cd8f03-12af-4ebd-b751-278123bb4390\") "
Oct 30 19:15:21.216467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.216311    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25cd8f03-12af-4ebd-b751-278123bb4390-kube-api-access-jbrcw\" (UniqueName: \"kubernetes.io/projected/25cd8f03-12af-4ebd-b751-278123bb4390-kube-api-access-jbrcw\") pod \"25cd8f03-12af-4ebd-b751-278123bb4390\" (UID: \"25cd8f03-12af-4ebd-b751-278123bb4390\") "
Oct 30 19:15:21.216575 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.216553    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25cd8f03-12af-4ebd-b751-278123bb4390-maas-api-service-ca" pod "25cd8f03-12af-4ebd-b751-278123bb4390" (UID: "25cd8f03-12af-4ebd-b751-278123bb4390"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:21.216909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.216891    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd8f03-12af-4ebd-b751-278123bb4390-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.218449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.218430    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25cd8f03-12af-4ebd-b751-278123bb4390-kube-api-access-jbrcw" pod "25cd8f03-12af-4ebd-b751-278123bb4390" (UID: "25cd8f03-12af-4ebd-b751-278123bb4390"). InnerVolumeSpecName "kube-api-access-jbrcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:21.318121 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.318087    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbrcw\" (UniqueName: \"kubernetes.io/projected/25cd8f03-12af-4ebd-b751-278123bb4390-kube-api-access-jbrcw\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.541192 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.541112    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" path="/var/lib/kubelet/pods/c25f605a-89a5-4b5b-b3ed-348bd91e541c/volumes"
Oct 30 19:15:21.972522 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.972459    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"
Oct 30 19:15:21.972522 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.972484    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74" event={"ID":"25cd8f03-12af-4ebd-b751-278123bb4390","Type":"ContainerDied","Data":"b56ca6ecbf3bb6f5ffb5d5a92df9fbe5f7427971338734d8784f19671418fe8d"}
Oct 30 19:15:21.973035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:21.972548    5232 scope.go:122] "RemoveContainer" containerID="ebf32d4f891375d19537766ac9335d9f900b54ec0dd37575feef41069dbb3278"
Oct 30 19:15:22.134455 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:22.134414    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"]
Oct 30 19:15:22.140139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:22.140110    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-22fb18-29889795-mlg74"]
Oct 30 19:15:23.541609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:23.541571    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" path="/var/lib/kubelet/pods/25cd8f03-12af-4ebd-b751-278123bb4390/volumes"
Oct 30 19:15:41.057467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:41.057430    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"]
Oct 30 19:15:41.116697 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:41.116658    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-060501da-8d4mc"]
Oct 30 19:15:41.542026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:15:41.541983    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40163aef-4889-4746-a224-e6d86aa36aa2" path="/var/lib/kubelet/pods/40163aef-4889-4746-a224-e6d86aa36aa2/volumes"
Oct 30 19:15:46.193602 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:15:46.193553    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:16:01.522007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:01.521962    5232 patch_prober.go:29] interesting pod/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:16:01.522523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:01.522042    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:16:03.374518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:03.374480    5232 generic.go:356] "Generic (PLEG): container finished" podID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerID="fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3" exitCode=0
Oct 30 19:16:03.375039 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:03.374512    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" event={"ID":"cff4f45b-6dbb-41e5-9a75-59aa00b8712e","Type":"ContainerDied","Data":"fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3"}
Oct 30 19:16:03.375092 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:03.375049    5232 scope.go:122] "RemoveContainer" containerID="fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3"
Oct 30 19:16:03.376729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:03.376697    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:16:03.376876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:03.376783    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:16:03.376876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:03.376853    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:16:04.383756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:04.383720    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" event={"ID":"cff4f45b-6dbb-41e5-9a75-59aa00b8712e","Type":"ContainerStarted","Data":"86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb"}
Oct 30 19:16:04.653408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:04.653252    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:16:04.653705 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:04.653681    5232 patch_prober.go:29] interesting pod/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:16:04.653896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:04.653870    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:16:05.659452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:05.659407    5232 patch_prober.go:29] interesting pod/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:16:05.660105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:05.660072    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:16:06.661997 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:06.661957    5232 patch_prober.go:29] interesting pod/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:16:06.662431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:06.662043    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:16:07.661959 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:07.661914    5232 patch_prober.go:29] interesting pod/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:16:07.662175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:07.661999    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:16:08.646845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:08.642154    5232 patch_prober.go:29] interesting pod/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:16:08.646845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:08.642216    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:16:09.645436 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:09.645402    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:16:09.645864 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:09.645844    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:16:09.646401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:09.646379    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:16:16.193937 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:16:16.193897    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:16:46.187601 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:16:46.187565    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:16:47.790541 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:47.790501    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:16:47.795395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:47.795376    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:16:47.802315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:47.802291    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:16:47.806434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:47.806412    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:16:48.187045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:48.187011    5232 scope.go:122] "RemoveContainer" containerID="f76c934e02260f9539e55e5a0f568fb486bb9d0c6a715873c0e14b503b137d85"
Oct 30 19:16:48.859731 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:48.859693    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"]
Oct 30 19:16:48.860177 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:48.859922    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy" containerID="cri-o://acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69" gracePeriod=30
Oct 30 19:16:50.412542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.412500    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"]
Oct 30 19:16:50.413002 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.412739    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt" podUID="f8d74b9a-50fb-429c-a202-c1ae9e782834" containerName="maas-api" containerID="cri-o://c94add83c5d153a5bf23d8665b714992fd9f36b95960fb989ba83ca02d2a66d1" gracePeriod=30
Oct 30 19:16:50.732700 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.732669    5232 generic.go:356] "Generic (PLEG): container finished" podID="f8d74b9a-50fb-429c-a202-c1ae9e782834" containerID="c94add83c5d153a5bf23d8665b714992fd9f36b95960fb989ba83ca02d2a66d1" exitCode=0
Oct 30 19:16:50.732876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.732739    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt" event={"ID":"f8d74b9a-50fb-429c-a202-c1ae9e782834","Type":"ContainerDied","Data":"c94add83c5d153a5bf23d8665b714992fd9f36b95960fb989ba83ca02d2a66d1"}
Oct 30 19:16:50.732876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.732793    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt" event={"ID":"f8d74b9a-50fb-429c-a202-c1ae9e782834","Type":"ContainerDied","Data":"1a5edddead674875f4bc2b5b67fa82fa832ddde8ca6f9ab3c169d34f8ecdbcce"}
Oct 30 19:16:50.732876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.732832    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a5edddead674875f4bc2b5b67fa82fa832ddde8ca6f9ab3c169d34f8ecdbcce"
Oct 30 19:16:50.743314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.743293    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:16:50.799422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.798516    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8d74b9a-50fb-429c-a202-c1ae9e782834-kube-api-access-4ghbv\" (UniqueName: \"kubernetes.io/projected/f8d74b9a-50fb-429c-a202-c1ae9e782834-kube-api-access-4ghbv\") pod \"f8d74b9a-50fb-429c-a202-c1ae9e782834\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") "
Oct 30 19:16:50.799422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.798616    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8d74b9a-50fb-429c-a202-c1ae9e782834-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d74b9a-50fb-429c-a202-c1ae9e782834-openshift-service-ca\") pod \"f8d74b9a-50fb-429c-a202-c1ae9e782834\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") "
Oct 30 19:16:50.799422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.798678    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls\") pod \"f8d74b9a-50fb-429c-a202-c1ae9e782834\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") "
Oct 30 19:16:50.799422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.798759    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-metrics-certs\") pod \"f8d74b9a-50fb-429c-a202-c1ae9e782834\" (UID: \"f8d74b9a-50fb-429c-a202-c1ae9e782834\") "
Oct 30 19:16:50.799422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.799232    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8d74b9a-50fb-429c-a202-c1ae9e782834-openshift-service-ca" pod "f8d74b9a-50fb-429c-a202-c1ae9e782834" (UID: "f8d74b9a-50fb-429c-a202-c1ae9e782834"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:50.800797 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.800774    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d74b9a-50fb-429c-a202-c1ae9e782834-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:50.802405 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.802381    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls" pod "f8d74b9a-50fb-429c-a202-c1ae9e782834" (UID: "f8d74b9a-50fb-429c-a202-c1ae9e782834"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:50.820886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.814176    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-metrics-certs" pod "f8d74b9a-50fb-429c-a202-c1ae9e782834" (UID: "f8d74b9a-50fb-429c-a202-c1ae9e782834"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:50.844940 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.844103    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d74b9a-50fb-429c-a202-c1ae9e782834-kube-api-access-4ghbv" pod "f8d74b9a-50fb-429c-a202-c1ae9e782834" (UID: "f8d74b9a-50fb-429c-a202-c1ae9e782834"). InnerVolumeSpecName "kube-api-access-4ghbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:50.902014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.901975    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:50.902014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.902012    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d74b9a-50fb-429c-a202-c1ae9e782834-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:50.902274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:50.902024    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ghbv\" (UniqueName: \"kubernetes.io/projected/f8d74b9a-50fb-429c-a202-c1ae9e782834-kube-api-access-4ghbv\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:51.740498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:51.740468    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"
Oct 30 19:16:51.902863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:51.902783    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"]
Oct 30 19:16:51.950508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:51.950475    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-58rlt"]
Oct 30 19:16:53.542533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:53.542493    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8d74b9a-50fb-429c-a202-c1ae9e782834" path="/var/lib/kubelet/pods/f8d74b9a-50fb-429c-a202-c1ae9e782834/volumes"
Oct 30 19:16:54.137701 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.137627    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:16:54.250587 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.250550    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0569546-94e1-4748-bcee-74965cf73a78-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0569546-94e1-4748-bcee-74965cf73a78-istiod-ca-cert\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.250587 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.250590    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-certs\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.250861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.250612    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c0569546-94e1-4748-bcee-74965cf73a78-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0569546-94e1-4748-bcee-74965cf73a78-istio-podinfo\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.250861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.250657    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-data\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.250861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.250706    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-credential-socket\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.250861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.250762    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-socket\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.250861 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.250855    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-envoy\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.251124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.250931    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-certs" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.251124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.251008    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-istio-token\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-istio-token\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.251124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.251047    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-kube-api-access-kxzss\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-kube-api-access-kxzss\") pod \"c0569546-94e1-4748-bcee-74965cf73a78\" (UID: \"c0569546-94e1-4748-bcee-74965cf73a78\") "
Oct 30 19:16:54.251124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.251007    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-data" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.251315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.251151    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-socket" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.251315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.251170    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0569546-94e1-4748-bcee-74965cf73a78-istiod-ca-cert" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:54.251315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.251264    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-credential-socket" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.253237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.253203    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0569546-94e1-4748-bcee-74965cf73a78-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.253385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.253239    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.253385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.253255    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.253385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.253281    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.253385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.253280    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-envoy" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.253385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.253295    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.253778 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.253746    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-istio-token" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:54.254944 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.254909    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0569546-94e1-4748-bcee-74965cf73a78-istio-podinfo" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:54.255099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.254954    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-kube-api-access-kxzss" pod "c0569546-94e1-4748-bcee-74965cf73a78" (UID: "c0569546-94e1-4748-bcee-74965cf73a78"). InnerVolumeSpecName "kube-api-access-kxzss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:54.354896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.354862    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0569546-94e1-4748-bcee-74965cf73a78-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.354896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.354894    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0569546-94e1-4748-bcee-74965cf73a78-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.355091 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.354906    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.355091 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.354919    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxzss\" (UniqueName: \"kubernetes.io/projected/c0569546-94e1-4748-bcee-74965cf73a78-kube-api-access-kxzss\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.764784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.764741    5232 generic.go:356] "Generic (PLEG): container finished" podID="c0569546-94e1-4748-bcee-74965cf73a78" containerID="acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69" exitCode=0
Oct 30 19:16:54.765274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.764822    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"
Oct 30 19:16:54.765274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.764852    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" event={"ID":"c0569546-94e1-4748-bcee-74965cf73a78","Type":"ContainerDied","Data":"acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69"}
Oct 30 19:16:54.765274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.764910    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm" event={"ID":"c0569546-94e1-4748-bcee-74965cf73a78","Type":"ContainerDied","Data":"9cf86b5af97fda614e431b268bde72e99c5c1dfbf23e80c114cdd0ffedc85155"}
Oct 30 19:16:54.765274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.764936    5232 scope.go:122] "RemoveContainer" containerID="acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69"
Oct 30 19:16:54.776627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.776604    5232 scope.go:122] "RemoveContainer" containerID="acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69"
Oct 30 19:16:54.776933 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:16:54.776913    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69\": container with ID starting with acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69 not found: ID does not exist" containerID="acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69"
Oct 30 19:16:54.777008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:54.776941    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69"} err="failed to get container status \"acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69\": rpc error: code = NotFound desc = could not find container \"acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69\": container with ID starting with acedda628bda26a67b7a32da627a5872f71e3b1032f3df76c02b3a391e151d69 not found: ID does not exist"
Oct 30 19:16:55.027553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:55.020899    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"]
Oct 30 19:16:55.060207 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:55.056531    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-tb8qm"]
Oct 30 19:16:55.540699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:16:55.540662    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0569546-94e1-4748-bcee-74965cf73a78" path="/var/lib/kubelet/pods/c0569546-94e1-4748-bcee-74965cf73a78/volumes"
Oct 30 19:17:16.188911 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:16.188874    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:17:33.157741 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:33.157706    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"]
Oct 30 19:17:33.158278 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:33.157948    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy" containerID="cri-o://668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020" gracePeriod=30
Oct 30 19:17:33.204488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:33.204449    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"]
Oct 30 19:17:33.204668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:33.204646    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy" containerID="cri-o://63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e" gracePeriod=30
Oct 30 19:17:34.327802 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:34.327765    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"]
Oct 30 19:17:34.328259 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:34.327999    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy" containerID="cri-o://a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b" gracePeriod=30
Oct 30 19:17:34.375194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:34.375163    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"]
Oct 30 19:17:34.375399 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:34.375380    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy" containerID="cri-o://e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6" gracePeriod=30
Oct 30 19:17:34.378990 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:34.378966    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"]
Oct 30 19:17:34.379171 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:34.379151    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy" containerID="cri-o://883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b" gracePeriod=30
Oct 30 19:17:34.422939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:34.422902    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"]
Oct 30 19:17:34.423179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:34.423145    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy" containerID="cri-o://c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec" gracePeriod=30
Oct 30 19:17:37.741232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:37.741197    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:17:37.741642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:37.741264    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:17:38.494630 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.494601    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:17:38.532241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.532207    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:17:38.587004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.586970    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-credential-socket\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.587176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587046    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-data\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.587176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587080    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istiod-ca-cert\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.587176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587107    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istiod-ca-cert\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.587176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587136    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-certs\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.587396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587195    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-kube-api-access-6m7xj\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-kube-api-access-6m7xj\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.587396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587225    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-token\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-token\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.587396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587278    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-envoy\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.587396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587282    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-credential-socket" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.587396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587357    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-data" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.587642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587457    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-podinfo\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.587642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587489    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istiod-ca-cert" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:38.587642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587487    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istiod-ca-cert" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:38.587890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587664    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-socket\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.587890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587717    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-credential-socket\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.587890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587807    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-certs" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.587890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587851    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-podinfo\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.588124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587895    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-token\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-token\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.588124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587928    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-socket\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.588124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587967    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-data\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.588124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.588012    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-envoy\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.588124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.588050    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-certs\") pod \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\" (UID: \"0a151a48-f41c-4126-ae9c-db8f0d9874a0\") "
Oct 30 19:17:38.588384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.588141    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-kube-api-access-v99wd\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-kube-api-access-v99wd\") pod \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\" (UID: \"28c34d0c-f7b9-45a9-8478-ca45c25dd632\") "
Oct 30 19:17:38.588567 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.587931    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-socket" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.588710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.588046    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-credential-socket" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.588844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.588359    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-data" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.588973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.588540    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-socket" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.589278 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.589117    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-certs" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.590474    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-token" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591228    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-envoy" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591274    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591292    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591302    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591311    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591324    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591336    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591345    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591353    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591361    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591369    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.591395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591377    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.592059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.591957    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-kube-api-access-6m7xj" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "kube-api-access-6m7xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:38.592059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.592051    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-podinfo" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:38.592131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.592057    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-kube-api-access-v99wd" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "kube-api-access-v99wd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:38.592131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.592076    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-token" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:38.593680 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.593655    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-envoy" pod "28c34d0c-f7b9-45a9-8478-ca45c25dd632" (UID: "28c34d0c-f7b9-45a9-8478-ca45c25dd632"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:38.593908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.593880    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-podinfo" pod "0a151a48-f41c-4126-ae9c-db8f0d9874a0" (UID: "0a151a48-f41c-4126-ae9c-db8f0d9874a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:38.693250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.693160    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.693250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.693196    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.693250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.693209    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28c34d0c-f7b9-45a9-8478-ca45c25dd632-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.693250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.693224    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v99wd\" (UniqueName: \"kubernetes.io/projected/28c34d0c-f7b9-45a9-8478-ca45c25dd632-kube-api-access-v99wd\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.693250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.693240    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m7xj\" (UniqueName: \"kubernetes.io/projected/0a151a48-f41c-4126-ae9c-db8f0d9874a0-kube-api-access-6m7xj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.693636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.693255    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:38.693636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:38.693269    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a151a48-f41c-4126-ae9c-db8f0d9874a0-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.168954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.168891    5232 generic.go:356] "Generic (PLEG): container finished" podID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerID="668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020" exitCode=0
Oct 30 19:17:39.169482 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.168981    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"
Oct 30 19:17:39.169482 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.168978    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" event={"ID":"28c34d0c-f7b9-45a9-8478-ca45c25dd632","Type":"ContainerDied","Data":"668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020"}
Oct 30 19:17:39.169482 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.169119    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48" event={"ID":"28c34d0c-f7b9-45a9-8478-ca45c25dd632","Type":"ContainerDied","Data":"0ca40d11116f0e26ed6b176493534b3581ac77dd4969f507cbb7086271d64e5a"}
Oct 30 19:17:39.169482 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.169152    5232 scope.go:122] "RemoveContainer" containerID="668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020"
Oct 30 19:17:39.171145 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.171124    5232 generic.go:356] "Generic (PLEG): container finished" podID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerID="63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e" exitCode=0
Oct 30 19:17:39.171258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.171184    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" event={"ID":"0a151a48-f41c-4126-ae9c-db8f0d9874a0","Type":"ContainerDied","Data":"63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e"}
Oct 30 19:17:39.171258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.171202    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"
Oct 30 19:17:39.171258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.171217    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj" event={"ID":"0a151a48-f41c-4126-ae9c-db8f0d9874a0","Type":"ContainerDied","Data":"8c38511dbdded2fd2a68e6fa2e18c190cf1ea3efed4df5dc12c1704f70f42719"}
Oct 30 19:17:39.196638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.196620    5232 scope.go:122] "RemoveContainer" containerID="668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020"
Oct 30 19:17:39.196970 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:39.196949    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020\": container with ID starting with 668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020 not found: ID does not exist" containerID="668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020"
Oct 30 19:17:39.197046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.196979    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020"} err="failed to get container status \"668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020\": rpc error: code = NotFound desc = could not find container \"668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020\": container with ID starting with 668d4006acc014d0eab34a9511d0290de569b7715a1e26960f4ce84c711b6020 not found: ID does not exist"
Oct 30 19:17:39.197046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.197001    5232 scope.go:122] "RemoveContainer" containerID="63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e"
Oct 30 19:17:39.210488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.210466    5232 scope.go:122] "RemoveContainer" containerID="63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e"
Oct 30 19:17:39.210904 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:39.210883    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e\": container with ID starting with 63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e not found: ID does not exist" containerID="63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e"
Oct 30 19:17:39.210979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.210911    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e"} err="failed to get container status \"63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e\": rpc error: code = NotFound desc = could not find container \"63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e\": container with ID starting with 63f4fbfcd31b337636a400bca95f5504dfa0286fb82f1c5f80243663521fa40e not found: ID does not exist"
Oct 30 19:17:39.330161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.330105    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"]
Oct 30 19:17:39.377411 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.377382    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-2fszj"]
Oct 30 19:17:39.446404 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.446371    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"]
Oct 30 19:17:39.449648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.449618    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-tgw48"]
Oct 30 19:17:39.542980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.542112    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" path="/var/lib/kubelet/pods/0a151a48-f41c-4126-ae9c-db8f0d9874a0/volumes"
Oct 30 19:17:39.542980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.542897    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" path="/var/lib/kubelet/pods/28c34d0c-f7b9-45a9-8478-ca45c25dd632/volumes"
Oct 30 19:17:39.668782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.668753    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:17:39.744494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.744448    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:17:39.786981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.786956    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:17:39.793097 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.793075    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:17:39.819253 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819154    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-istio-token\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-istio-token\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.819253 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819215    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af8fe62f-77ae-424a-94c6-54df1663d850-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af8fe62f-77ae-424a-94c6-54df1663d850-istiod-ca-cert\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.819253 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819251    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-kube-api-access-7xkmk\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-kube-api-access-7xkmk\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819285    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/af8fe62f-77ae-424a-94c6-54df1663d850-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af8fe62f-77ae-424a-94c6-54df1663d850-istio-podinfo\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819371    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-socket\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819429    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-istio-token\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-istio-token\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819472    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-data\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819505    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-kube-api-access-95nkb\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-kube-api-access-95nkb\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819534    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e46471e-6120-431a-bd06-666deece5f65-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e46471e-6120-431a-bd06-666deece5f65-istiod-ca-cert\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819563    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-certs\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819683    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-envoy\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.819719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819675    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af8fe62f-77ae-424a-94c6-54df1663d850-istiod-ca-cert" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:39.820722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819741    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-data\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.820722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819794    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-credential-socket\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.820722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819881    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-envoy\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.820722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819907    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-socket\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.820722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.819953    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-certs\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.820722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.820009    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e46471e-6120-431a-bd06-666deece5f65-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e46471e-6120-431a-bd06-666deece5f65-istio-podinfo\") pod \"5e46471e-6120-431a-bd06-666deece5f65\" (UID: \"5e46471e-6120-431a-bd06-666deece5f65\") "
Oct 30 19:17:39.820722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.820589    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-credential-socket\") pod \"af8fe62f-77ae-424a-94c6-54df1663d850\" (UID: \"af8fe62f-77ae-424a-94c6-54df1663d850\") "
Oct 30 19:17:39.821436 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.821305    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af8fe62f-77ae-424a-94c6-54df1663d850-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.822290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.821518    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-credential-socket" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.822290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.821696    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-data" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.825123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.825093    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-istio-token" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.826146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.826100    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-istio-token" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.826297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.826237    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-kube-api-access-95nkb" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "kube-api-access-95nkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.827234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.827068    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e46471e-6120-431a-bd06-666deece5f65-istiod-ca-cert" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:39.827416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.827393    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-certs" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.828146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.828117    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-socket" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.828376 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.828355    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-kube-api-access-7xkmk" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "kube-api-access-7xkmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.828652 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.828629    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-certs" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.828874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.828852    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-socket" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.829363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.829340    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-data" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.829455 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.829367    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-credential-socket" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.831985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.831963    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e46471e-6120-431a-bd06-666deece5f65-istio-podinfo" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:39.832118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.832077    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-envoy" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.832471 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.832442    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af8fe62f-77ae-424a-94c6-54df1663d850-istio-podinfo" pod "af8fe62f-77ae-424a-94c6-54df1663d850" (UID: "af8fe62f-77ae-424a-94c6-54df1663d850"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:39.833373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.833349    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-envoy" pod "5e46471e-6120-431a-bd06-666deece5f65" (UID: "5e46471e-6120-431a-bd06-666deece5f65"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.922650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.922601    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5745ae03-a721-4681-80bc-369dc9932f96-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5745ae03-a721-4681-80bc-369dc9932f96-istiod-ca-cert\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.922981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.922957    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-istio-token\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-istio-token\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.923122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.923101    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/74805600-c429-41e2-8974-5c9c42079692-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74805600-c429-41e2-8974-5c9c42079692-istio-podinfo\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.923259 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.923244    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-socket\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.923380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.923116    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5745ae03-a721-4681-80bc-369dc9932f96-istiod-ca-cert" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:39.923449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.923366    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-data\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.923773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.923737    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-socket" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.923902 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.923450    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-kube-api-access-rk9qk\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-kube-api-access-rk9qk\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.923994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.923971    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-data\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.924061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924032    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-socket\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.924125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924062    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-envoy\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.924181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924107    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74805600-c429-41e2-8974-5c9c42079692-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74805600-c429-41e2-8974-5c9c42079692-istiod-ca-cert\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.924275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924253    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-data" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.924419 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924397    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-credential-socket\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.924498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924426    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-socket" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.924565 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924499    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-certs\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.924625 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924579    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-credential-socket\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.924673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924625    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-istio-token\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-istio-token\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.924673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924639    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-data" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.924783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924660    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-certs\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.924783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924714    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5745ae03-a721-4681-80bc-369dc9932f96-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5745ae03-a721-4681-80bc-369dc9932f96-istio-podinfo\") pod \"5745ae03-a721-4681-80bc-369dc9932f96\" (UID: \"5745ae03-a721-4681-80bc-369dc9932f96\") "
Oct 30 19:17:39.924783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924723    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74805600-c429-41e2-8974-5c9c42079692-istiod-ca-cert" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:39.924783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924741    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-envoy\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.925040 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924953    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-certs" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.925040 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.924967    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-kube-api-access-lds6n\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-kube-api-access-lds6n\") pod \"74805600-c429-41e2-8974-5c9c42079692\" (UID: \"74805600-c429-41e2-8974-5c9c42079692\") "
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925287    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-credential-socket" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925639    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925663    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925679    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5745ae03-a721-4681-80bc-369dc9932f96-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925694    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925708    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95nkb\" (UniqueName: \"kubernetes.io/projected/af8fe62f-77ae-424a-94c6-54df1663d850-kube-api-access-95nkb\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925725    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e46471e-6120-431a-bd06-666deece5f65-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925739    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925753    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925767    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925781    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925795    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925828    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925842    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74805600-c429-41e2-8974-5c9c42079692-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925857    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925871    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925885    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925900    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925914    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925928    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e46471e-6120-431a-bd06-666deece5f65-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925945    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925979    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e46471e-6120-431a-bd06-666deece5f65-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.925994    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af8fe62f-77ae-424a-94c6-54df1663d850-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.926008    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.926022    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xkmk\" (UniqueName: \"kubernetes.io/projected/5e46471e-6120-431a-bd06-666deece5f65-kube-api-access-7xkmk\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.926036    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af8fe62f-77ae-424a-94c6-54df1663d850-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.926302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.926269    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-credential-socket" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.927726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.926403    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-certs" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.928005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.927956    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-kube-api-access-rk9qk" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "kube-api-access-rk9qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.928314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.928290    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5745ae03-a721-4681-80bc-369dc9932f96-istio-podinfo" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:39.928402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.928377    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-envoy" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.929246 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.929211    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-kube-api-access-lds6n" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "kube-api-access-lds6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.929385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.929358    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-istio-token" pod "5745ae03-a721-4681-80bc-369dc9932f96" (UID: "5745ae03-a721-4681-80bc-369dc9932f96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.929640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.929623    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-envoy" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.929954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.929924    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-istio-token" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.931059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:39.931038    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74805600-c429-41e2-8974-5c9c42079692-istio-podinfo" pod "74805600-c429-41e2-8974-5c9c42079692" (UID: "74805600-c429-41e2-8974-5c9c42079692"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:40.027989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.027914    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.027989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.027946    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.027989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.027955    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.027989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.027964    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5745ae03-a721-4681-80bc-369dc9932f96-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.027989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.027973    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74805600-c429-41e2-8974-5c9c42079692-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.027989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.027981    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lds6n\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-kube-api-access-lds6n\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.027989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.027992    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74805600-c429-41e2-8974-5c9c42079692-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.028317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.028000    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74805600-c429-41e2-8974-5c9c42079692-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.028317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.028009    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk9qk\" (UniqueName: \"kubernetes.io/projected/5745ae03-a721-4681-80bc-369dc9932f96-kube-api-access-rk9qk\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.028317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.028017    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5745ae03-a721-4681-80bc-369dc9932f96-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.217079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.217040    5232 generic.go:356] "Generic (PLEG): container finished" podID="af8fe62f-77ae-424a-94c6-54df1663d850" containerID="e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6" exitCode=0
Oct 30 19:17:40.217745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.217103    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" event={"ID":"af8fe62f-77ae-424a-94c6-54df1663d850","Type":"ContainerDied","Data":"e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6"}
Oct 30 19:17:40.217745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.217157    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk" event={"ID":"af8fe62f-77ae-424a-94c6-54df1663d850","Type":"ContainerDied","Data":"5e874c1690b14d918bd0d2cacefb29958925266e8299ef22840e2087d82ab2ed"}
Oct 30 19:17:40.217745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.217187    5232 scope.go:122] "RemoveContainer" containerID="e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6"
Oct 30 19:17:40.218041 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.218019    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.223534    5232 generic.go:356] "Generic (PLEG): container finished" podID="5e46471e-6120-431a-bd06-666deece5f65" containerID="a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b" exitCode=0
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.223663    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" event={"ID":"5e46471e-6120-431a-bd06-666deece5f65","Type":"ContainerDied","Data":"a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b"}
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.223706    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4" event={"ID":"5e46471e-6120-431a-bd06-666deece5f65","Type":"ContainerDied","Data":"b4db9d59255b48fc8ad363001a9795ce98a5a8ee49dd4b280f38de8bb523892c"}
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.223788    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.234667    5232 generic.go:356] "Generic (PLEG): container finished" podID="5745ae03-a721-4681-80bc-369dc9932f96" containerID="c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec" exitCode=0
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.234851    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" event={"ID":"5745ae03-a721-4681-80bc-369dc9932f96","Type":"ContainerDied","Data":"c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec"}
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.234926    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29" event={"ID":"5745ae03-a721-4681-80bc-369dc9932f96","Type":"ContainerDied","Data":"3e26dc413cb34c2b7f3a57578bb3ed6c4a99db3a56a235d643c6b7769e296f42"}
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.243585    5232 generic.go:356] "Generic (PLEG): container finished" podID="74805600-c429-41e2-8974-5c9c42079692" containerID="883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b" exitCode=0
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.243675    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" event={"ID":"74805600-c429-41e2-8974-5c9c42079692","Type":"ContainerDied","Data":"883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b"}
Oct 30 19:17:40.247128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.243720    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4" event={"ID":"74805600-c429-41e2-8974-5c9c42079692","Type":"ContainerDied","Data":"e8c7f761f729dfbf576ad35f70ac2de3a8ddb1310cfe666522b63f5660a0facb"}
Oct 30 19:17:40.249267 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.247886    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"
Oct 30 19:17:40.249549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.249524    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"
Oct 30 19:17:40.262984 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.262961    5232 scope.go:122] "RemoveContainer" containerID="e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6"
Oct 30 19:17:40.263543 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:40.263518    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6\": container with ID starting with e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6 not found: ID does not exist" containerID="e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6"
Oct 30 19:17:40.263624 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.263555    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6"} err="failed to get container status \"e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6\": rpc error: code = NotFound desc = could not find container \"e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6\": container with ID starting with e202f9dc8acd35daefd72ce54198f3f51a095987dbbc130af2d7e33dc4384cb6 not found: ID does not exist"
Oct 30 19:17:40.263624 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.263576    5232 scope.go:122] "RemoveContainer" containerID="a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b"
Oct 30 19:17:40.278198 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.278140    5232 scope.go:122] "RemoveContainer" containerID="a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b"
Oct 30 19:17:40.278725 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:40.278663    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b\": container with ID starting with a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b not found: ID does not exist" containerID="a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b"
Oct 30 19:17:40.278725 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.278704    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b"} err="failed to get container status \"a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b\": rpc error: code = NotFound desc = could not find container \"a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b\": container with ID starting with a9349d71dfa16be98d6733b93b4c1fa78e309311926d778ef87673cdbea4ef9b not found: ID does not exist"
Oct 30 19:17:40.278960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.278731    5232 scope.go:122] "RemoveContainer" containerID="c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec"
Oct 30 19:17:40.291869 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.291808    5232 scope.go:122] "RemoveContainer" containerID="9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19"
Oct 30 19:17:40.304784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.304759    5232 scope.go:122] "RemoveContainer" containerID="c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec"
Oct 30 19:17:40.305114 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:40.305091    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec\": container with ID starting with c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec not found: ID does not exist" containerID="c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec"
Oct 30 19:17:40.305209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.305122    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec"} err="failed to get container status \"c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec\": rpc error: code = NotFound desc = could not find container \"c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec\": container with ID starting with c1a160d16432d9a08c868a0db13b129936e3a7d7382bf1c542e69e17c0df98ec not found: ID does not exist"
Oct 30 19:17:40.305209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.305147    5232 scope.go:122] "RemoveContainer" containerID="9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19"
Oct 30 19:17:40.305470 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:40.305443    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19\": container with ID starting with 9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19 not found: ID does not exist" containerID="9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19"
Oct 30 19:17:40.305530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.305482    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19"} err="failed to get container status \"9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19\": rpc error: code = NotFound desc = could not find container \"9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19\": container with ID starting with 9cdcbf5f629b0e0c5f469a66632cf5f689a2019b3eb649d5215a6127dadbeb19 not found: ID does not exist"
Oct 30 19:17:40.305530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.305507    5232 scope.go:122] "RemoveContainer" containerID="883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b"
Oct 30 19:17:40.317557 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.317533    5232 scope.go:122] "RemoveContainer" containerID="883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b"
Oct 30 19:17:40.317871 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:40.317849    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b\": container with ID starting with 883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b not found: ID does not exist" containerID="883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b"
Oct 30 19:17:40.317939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.317879    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b"} err="failed to get container status \"883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b\": rpc error: code = NotFound desc = could not find container \"883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b\": container with ID starting with 883ab8ad24e7504c2de49fc6532e9ba9dc86e29c3957a8e2b3fccde63abaee9b not found: ID does not exist"
Oct 30 19:17:40.380020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.379987    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"]
Oct 30 19:17:40.388645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.388618    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-zsj29"]
Oct 30 19:17:40.503520 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.503487    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"]
Oct 30 19:17:40.506534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.506512    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vt7wk"]
Oct 30 19:17:40.611706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.611535    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"]
Oct 30 19:17:40.615273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.615209    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-bdzb4"]
Oct 30 19:17:40.723072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.723033    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"]
Oct 30 19:17:40.733877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:40.733841    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-hqdb4"]
Oct 30 19:17:41.540315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:41.540279    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5745ae03-a721-4681-80bc-369dc9932f96" path="/var/lib/kubelet/pods/5745ae03-a721-4681-80bc-369dc9932f96/volumes"
Oct 30 19:17:41.540763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:41.540749    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e46471e-6120-431a-bd06-666deece5f65" path="/var/lib/kubelet/pods/5e46471e-6120-431a-bd06-666deece5f65/volumes"
Oct 30 19:17:41.541188 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:41.541175    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74805600-c429-41e2-8974-5c9c42079692" path="/var/lib/kubelet/pods/74805600-c429-41e2-8974-5c9c42079692/volumes"
Oct 30 19:17:41.541926 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:41.541911    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" path="/var/lib/kubelet/pods/af8fe62f-77ae-424a-94c6-54df1663d850/volumes"
Oct 30 19:17:46.190840 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:46.190795    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:17:46.666389 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:46.666348    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"]
Oct 30 19:17:46.666620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:46.666586    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy" containerID="cri-o://af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb" gracePeriod=30
Oct 30 19:17:46.717109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:46.717070    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"]
Oct 30 19:17:46.717624 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:46.717321    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy" containerID="cri-o://3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492" gracePeriod=30
Oct 30 19:17:46.720472 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:46.720441    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"]
Oct 30 19:17:46.720717 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:46.720676    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy" containerID="cri-o://e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1" gracePeriod=30
Oct 30 19:17:46.761177 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:46.761145    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"]
Oct 30 19:17:46.761387 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:46.761363    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy" containerID="cri-o://abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b" gracePeriod=30
Oct 30 19:17:48.411481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:48.411433    5232 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:17:48.411996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:48.411514    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:17:52.019703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.019669    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.071695    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-data\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.071752    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-credential-socket\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.071947    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-envoy\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072026    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-certs\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072063    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-data" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072079    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66491f90-bac7-4ddf-8452-6471424bc4af-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66491f90-bac7-4ddf-8452-6471424bc4af-istiod-ca-cert\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072080    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-credential-socket" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072187    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-istio-token\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-istio-token\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072226    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-kube-api-access-494gt\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-kube-api-access-494gt\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072256    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/66491f90-bac7-4ddf-8452-6471424bc4af-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66491f90-bac7-4ddf-8452-6471424bc4af-istio-podinfo\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072284    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-certs" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072301    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-socket\") pod \"66491f90-bac7-4ddf-8452-6471424bc4af\" (UID: \"66491f90-bac7-4ddf-8452-6471424bc4af\") "
Oct 30 19:17:52.072722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072530    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66491f90-bac7-4ddf-8452-6471424bc4af-istiod-ca-cert" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:52.073526 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072959    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.073526 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072981    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66491f90-bac7-4ddf-8452-6471424bc4af-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.073526 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.072994    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.073526 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.073033    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.073807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.073772    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-socket" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.076037 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.076010    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-kube-api-access-494gt" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "kube-api-access-494gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.076142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.076008    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-envoy" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.078586 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.078558    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-istio-token" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.078741 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.078664    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66491f90-bac7-4ddf-8452-6471424bc4af-istio-podinfo" pod "66491f90-bac7-4ddf-8452-6471424bc4af" (UID: "66491f90-bac7-4ddf-8452-6471424bc4af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:52.153831 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.134251    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175295    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-socket\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175386    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-token\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-token\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175456    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-data\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175489    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-kube-api-access-rj25m\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-kube-api-access-rj25m\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175524    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-envoy\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175580    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-podinfo\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175627    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-certs\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175755    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-credential-socket\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.182373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.175800    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istiod-ca-cert\") pod \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\" (UID: \"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729\") "
Oct 30 19:17:52.183527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.183126    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.183527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.183180    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.183527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.183199    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-494gt\" (UniqueName: \"kubernetes.io/projected/66491f90-bac7-4ddf-8452-6471424bc4af-kube-api-access-494gt\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.183527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.183225    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66491f90-bac7-4ddf-8452-6471424bc4af-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.183527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.183249    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66491f90-bac7-4ddf-8452-6471424bc4af-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.184294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.184034    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istiod-ca-cert" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:52.184744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.184414    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-socket" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.187573 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.187384    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-token" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.188018 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.187699    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-data" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.189216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.189046    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-certs" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.189449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.189413    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-credential-socket" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.193262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.193220    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-podinfo" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:52.193543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.193483    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-kube-api-access-rj25m" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "kube-api-access-rj25m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.205451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.205399    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-envoy" pod "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" (UID: "f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.272934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.272904    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:17:52.282214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.282162    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:17:52.285927 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.285899    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.286038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.285931    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.286038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.285949    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.286038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.285963    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.286038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.285978    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj25m\" (UniqueName: \"kubernetes.io/projected/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-kube-api-access-rj25m\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.286038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.285994    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.286038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.286007    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.286038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.286022    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.286038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.286035    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.350514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.350462    5232 generic.go:356] "Generic (PLEG): container finished" podID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerID="e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1" exitCode=0
Oct 30 19:17:52.350684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.350565    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" event={"ID":"9fe130c6-99a4-4c49-9e73-d364b9239099","Type":"ContainerDied","Data":"e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1"}
Oct 30 19:17:52.350684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.350621    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv" event={"ID":"9fe130c6-99a4-4c49-9e73-d364b9239099","Type":"ContainerDied","Data":"947caa0208969ff726fc143ea4ea2300bdfa9da2643f3b35570d2ac09b6a8e3a"}
Oct 30 19:17:52.350684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.350652    5232 scope.go:122] "RemoveContainer" containerID="e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1"
Oct 30 19:17:52.350952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.350842    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"
Oct 30 19:17:52.353991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.353806    5232 generic.go:356] "Generic (PLEG): container finished" podID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerID="abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b" exitCode=0
Oct 30 19:17:52.353991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.353985    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"
Oct 30 19:17:52.357290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.357255    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" event={"ID":"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729","Type":"ContainerDied","Data":"abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b"}
Oct 30 19:17:52.357406 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.357321    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d" event={"ID":"f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729","Type":"ContainerDied","Data":"6476de0dd1d1e080bc9160cecbc8351c207f72a6a34b0b93521ce418b65085c3"}
Oct 30 19:17:52.359406 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.359379    5232 generic.go:356] "Generic (PLEG): container finished" podID="66491f90-bac7-4ddf-8452-6471424bc4af" containerID="af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb" exitCode=0
Oct 30 19:17:52.359602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.359439    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"
Oct 30 19:17:52.359910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.359449    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" event={"ID":"66491f90-bac7-4ddf-8452-6471424bc4af","Type":"ContainerDied","Data":"af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb"}
Oct 30 19:17:52.359910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.359855    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm" event={"ID":"66491f90-bac7-4ddf-8452-6471424bc4af","Type":"ContainerDied","Data":"b334be42413cd4ec7a9dcaf5d4b204cbab458a31dc2fd060969bce7a9caa1f40"}
Oct 30 19:17:52.361598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.361570    5232 generic.go:356] "Generic (PLEG): container finished" podID="762a7039-237a-4de1-a8a6-af940f2581d3" containerID="3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492" exitCode=0
Oct 30 19:17:52.361704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.361678    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" event={"ID":"762a7039-237a-4de1-a8a6-af940f2581d3","Type":"ContainerDied","Data":"3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492"}
Oct 30 19:17:52.361780 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.361715    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb" event={"ID":"762a7039-237a-4de1-a8a6-af940f2581d3","Type":"ContainerDied","Data":"21c38535a29f62e0e2468772f7b51666e632a4b08af89e2f28c8f2bc341d28a8"}
Oct 30 19:17:52.361875 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.361783    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"
Oct 30 19:17:52.375556 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.375496    5232 scope.go:122] "RemoveContainer" containerID="e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1"
Oct 30 19:17:52.375939 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:52.375908    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1\": container with ID starting with e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1 not found: ID does not exist" containerID="e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1"
Oct 30 19:17:52.376028 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.375951    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1"} err="failed to get container status \"e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1\": rpc error: code = NotFound desc = could not find container \"e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1\": container with ID starting with e3effc2e31916485ccb0feb4608991b2aa8c3281081b82a0f4b259db01f538c1 not found: ID does not exist"
Oct 30 19:17:52.376028 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.375977    5232 scope.go:122] "RemoveContainer" containerID="abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b"
Oct 30 19:17:52.386879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.386805    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/762a7039-237a-4de1-a8a6-af940f2581d3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/762a7039-237a-4de1-a8a6-af940f2581d3-istio-podinfo\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.386915    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/762a7039-237a-4de1-a8a6-af940f2581d3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/762a7039-237a-4de1-a8a6-af940f2581d3-istiod-ca-cert\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.386948    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-credential-socket\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.386973    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-envoy\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.387221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387045    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-socket\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387086    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-kube-api-access-dx4jr\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-kube-api-access-dx4jr\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387129    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-kube-api-access-fmcsv\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-kube-api-access-fmcsv\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.387221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387173    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-podinfo\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.387417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387237    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-certs\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387259    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-envoy\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387281    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-credential-socket\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.387417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387301    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-istio-token\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-istio-token\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387344    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fe130c6-99a4-4c49-9e73-d364b9239099-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe130c6-99a4-4c49-9e73-d364b9239099-istiod-ca-cert\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.387417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387365    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-socket\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.387417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387408    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-data\") pod \"762a7039-237a-4de1-a8a6-af940f2581d3\" (UID: \"762a7039-237a-4de1-a8a6-af940f2581d3\") "
Oct 30 19:17:52.387756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387427    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-token\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-token\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.387756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387467    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-certs\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.387756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.387492    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-data\") pod \"9fe130c6-99a4-4c49-9e73-d364b9239099\" (UID: \"9fe130c6-99a4-4c49-9e73-d364b9239099\") "
Oct 30 19:17:52.388100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.388078    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-data" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.388597    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-socket" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.388613    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-certs" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.388712    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe130c6-99a4-4c49-9e73-d364b9239099-istiod-ca-cert" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.388955    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/762a7039-237a-4de1-a8a6-af940f2581d3-istiod-ca-cert" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.389204    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-credential-socket" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.390173    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-socket" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.390393    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-data" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.390590    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-certs" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.390949    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/762a7039-237a-4de1-a8a6-af940f2581d3-istio-podinfo" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:52.391242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.390961    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-credential-socket" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.391870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.391579    5232 scope.go:122] "RemoveContainer" containerID="abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b"
Oct 30 19:17:52.392063 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:52.392032    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b\": container with ID starting with abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b not found: ID does not exist" containerID="abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b"
Oct 30 19:17:52.392157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.392077    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b"} err="failed to get container status \"abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b\": rpc error: code = NotFound desc = could not find container \"abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b\": container with ID starting with abe6149787a503ddfafd9e9f0637c0155016397aadd2ea4f159029b1862e483b not found: ID does not exist"
Oct 30 19:17:52.392157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.392108    5232 scope.go:122] "RemoveContainer" containerID="af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb"
Oct 30 19:17:52.395281 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.395202    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-istio-token" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.395401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.395287    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-kube-api-access-fmcsv" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "kube-api-access-fmcsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.395649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.395616    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-token" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.396912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.396797    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-podinfo" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:52.397233 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.397214    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-envoy" pod "9fe130c6-99a4-4c49-9e73-d364b9239099" (UID: "9fe130c6-99a4-4c49-9e73-d364b9239099"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.399074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.399052    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-kube-api-access-dx4jr" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "kube-api-access-dx4jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.400217 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.400193    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-envoy" pod "762a7039-237a-4de1-a8a6-af940f2581d3" (UID: "762a7039-237a-4de1-a8a6-af940f2581d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:52.403482 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.403460    5232 scope.go:122] "RemoveContainer" containerID="af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb"
Oct 30 19:17:52.403799 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:52.403782    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb\": container with ID starting with af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb not found: ID does not exist" containerID="af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb"
Oct 30 19:17:52.403900 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.403806    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb"} err="failed to get container status \"af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb\": rpc error: code = NotFound desc = could not find container \"af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb\": container with ID starting with af5905cb42d138d8a7ab9ef4877f0bcc387e8800e53f49c723f47eb2e19cc7bb not found: ID does not exist"
Oct 30 19:17:52.403900 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.403894    5232 scope.go:122] "RemoveContainer" containerID="3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492"
Oct 30 19:17:52.413156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.413129    5232 scope.go:122] "RemoveContainer" containerID="3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492"
Oct 30 19:17:52.413487 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:17:52.413465    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492\": container with ID starting with 3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492 not found: ID does not exist" containerID="3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492"
Oct 30 19:17:52.413533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.413504    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492"} err="failed to get container status \"3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492\": rpc error: code = NotFound desc = could not find container \"3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492\": container with ID starting with 3a7e66875a3a15b20fda89ed8d448cf8d9b2c4dbf710b36e72a14e99b5812492 not found: ID does not exist"
Oct 30 19:17:52.488999 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.488962    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.488999 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.488998    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx4jr\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-kube-api-access-dx4jr\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489014    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmcsv\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-kube-api-access-fmcsv\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489023    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489032    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489040    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489049    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489059    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/762a7039-237a-4de1-a8a6-af940f2581d3-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489070    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe130c6-99a4-4c49-9e73-d364b9239099-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489083    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489091    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489100    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489109    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489118    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489126    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/762a7039-237a-4de1-a8a6-af940f2581d3-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489135    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/762a7039-237a-4de1-a8a6-af940f2581d3-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489143    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/762a7039-237a-4de1-a8a6-af940f2581d3-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.489333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.489156    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe130c6-99a4-4c49-9e73-d364b9239099-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.747975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.747936    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"]
Oct 30 19:17:52.751881 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.751845    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-mmn8d"]
Oct 30 19:17:52.912237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.912148    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"]
Oct 30 19:17:52.915305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:52.915277    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-rflgm"]
Oct 30 19:17:53.020990 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:53.020944    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"]
Oct 30 19:17:53.026465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:53.026432    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-sngwb"]
Oct 30 19:17:53.132720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:53.132683    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"]
Oct 30 19:17:53.139479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:53.139436    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-svhhv"]
Oct 30 19:17:53.542559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:53.542512    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" path="/var/lib/kubelet/pods/66491f90-bac7-4ddf-8452-6471424bc4af/volumes"
Oct 30 19:17:53.543171 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:53.543149    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" path="/var/lib/kubelet/pods/762a7039-237a-4de1-a8a6-af940f2581d3/volumes"
Oct 30 19:17:53.543762 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:53.543742    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" path="/var/lib/kubelet/pods/9fe130c6-99a4-4c49-9e73-d364b9239099/volumes"
Oct 30 19:17:53.544782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:17:53.544759    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" path="/var/lib/kubelet/pods/f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729/volumes"
Oct 30 19:18:16.188832 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:18:16.188779    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:18:32.528288 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:32.528194    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"]
Oct 30 19:18:32.528785 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:32.528480    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy" containerID="cri-o://8e6b163f60eecb9211cae81f66b17ce081a414d507005b9b50536ff01126952f" gracePeriod=30
Oct 30 19:18:37.720774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.720732    5232 generic.go:356] "Generic (PLEG): container finished" podID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerID="8e6b163f60eecb9211cae81f66b17ce081a414d507005b9b50536ff01126952f" exitCode=0
Oct 30 19:18:37.721284 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.720889    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" event={"ID":"3d23bf28-1715-4699-b69f-ff91ff6dcfb4","Type":"ContainerDied","Data":"8e6b163f60eecb9211cae81f66b17ce081a414d507005b9b50536ff01126952f"}
Oct 30 19:18:37.818464 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.818393    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:18:37.937359 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937322    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-certs\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.937359 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937372    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-kube-api-access-cs4gq\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-kube-api-access-cs4gq\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.937608 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937443    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-credential-socket\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.937608 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937481    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-token\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-token\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.937711 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937616    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-data\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.937711 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937683    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-certs" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:37.937855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937735    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istiod-ca-cert\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.937855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937768    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-credential-socket" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:37.937855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937795    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-podinfo\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.938016 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937882    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-data" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:37.938016 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937896    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-envoy\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.938016 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.937959    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-socket\") pod \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\" (UID: \"3d23bf28-1715-4699-b69f-ff91ff6dcfb4\") "
Oct 30 19:18:37.938153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.938131    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istiod-ca-cert" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:37.938290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.938258    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-socket" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:37.938520 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.938500    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:37.938520 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.938522    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:37.938688 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.938535    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:37.938688 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.938549    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:37.938688 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.938562    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:37.939998 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.939972    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-kube-api-access-cs4gq" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "kube-api-access-cs4gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:37.940109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.940002    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-token" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:37.940303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.940281    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-envoy" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:37.940381 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:37.940297    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-podinfo" pod "3d23bf28-1715-4699-b69f-ff91ff6dcfb4" (UID: "3d23bf28-1715-4699-b69f-ff91ff6dcfb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:38.039639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.039603    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:38.039639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.039641    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:38.039916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.039656    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs4gq\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-kube-api-access-cs4gq\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:38.039916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.039672    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d23bf28-1715-4699-b69f-ff91ff6dcfb4-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:38.729045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.729003    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd" event={"ID":"3d23bf28-1715-4699-b69f-ff91ff6dcfb4","Type":"ContainerDied","Data":"71ec67347fdfb628e9784b51eb64edb17edd04ff67e6f9228c33098adb73510c"}
Oct 30 19:18:38.729419 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.729093    5232 scope.go:122] "RemoveContainer" containerID="8e6b163f60eecb9211cae81f66b17ce081a414d507005b9b50536ff01126952f"
Oct 30 19:18:38.729419 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.729232    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"
Oct 30 19:18:38.890319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.890243    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"]
Oct 30 19:18:38.937719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:38.937682    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-86lbd"]
Oct 30 19:18:39.516176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516135    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"]
Oct 30 19:18:39.516734 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516716    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:39.516789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516736    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:39.516789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516750    5232 cpu_manager.go:414] "containerMap: removing container" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy"
Oct 30 19:18:39.516789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516757    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy"
Oct 30 19:18:39.516789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516768    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy"
Oct 30 19:18:39.516789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516788    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516797    5232 cpu_manager.go:414] "containerMap: removing container" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516803    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516822    5232 cpu_manager.go:414] "containerMap: removing container" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516827    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516833    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516838    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516845    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516849    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516858    5232 cpu_manager.go:414] "containerMap: removing container" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516862    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516871    5232 cpu_manager.go:414] "containerMap: removing container" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516875    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516881    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516886    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516892    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f8d74b9a-50fb-429c-a202-c1ae9e782834" containerName="maas-api"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516896    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8d74b9a-50fb-429c-a202-c1ae9e782834" containerName="maas-api"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516905    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516910    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516918    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516923    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516929    5232 cpu_manager.go:414] "containerMap: removing container" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516935    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516945    5232 cpu_manager.go:414] "containerMap: removing container" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516949    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516955    5232 cpu_manager.go:414] "containerMap: removing container" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516959    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516967    5232 cpu_manager.go:414] "containerMap: removing container" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516971    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516977    5232 cpu_manager.go:414] "containerMap: removing container" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy"
Oct 30 19:18:39.516971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516982    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516988    5232 cpu_manager.go:414] "containerMap: removing container" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.516992    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517006    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517010    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517019    5232 cpu_manager.go:414] "containerMap: removing container" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517023    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517036    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517040    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517048    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517052    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517059    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517063    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517151    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0569546-94e1-4748-bcee-74965cf73a78" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517157    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517163    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517170    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517176    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="762a7039-237a-4de1-a8a6-af940f2581d3" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517183    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7d5ea4c-0972-4f5a-9cbe-dbf9da2b7729" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517188    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fe130c6-99a4-4c49-9e73-d364b9239099" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517193    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517197    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517203    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="74805600-c429-41e2-8974-5c9c42079692" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517210    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517215    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a151a48-f41c-4126-ae9c-db8f0d9874a0" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517221    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8d74b9a-50fb-429c-a202-c1ae9e782834" containerName="maas-api"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517226    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517231    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5745ae03-a721-4681-80bc-369dc9932f96" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517237    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="28c34d0c-f7b9-45a9-8478-ca45c25dd632" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517242    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517248    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="af8fe62f-77ae-424a-94c6-54df1663d850" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517253    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="66491f90-bac7-4ddf-8452-6471424bc4af" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517259    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517266    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e46471e-6120-431a-bd06-666deece5f65" containerName="istio-proxy"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517271    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c3b7657-e14f-4f73-b80a-ea8215c51157" containerName="cleanup"
Oct 30 19:18:39.517840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.517278    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:39.521563 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.521545    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.545601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.545526    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d23bf28-1715-4699-b69f-ff91ff6dcfb4" path="/var/lib/kubelet/pods/3d23bf28-1715-4699-b69f-ff91ff6dcfb4/volumes"
Oct 30 19:18:39.604453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.604408    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-905566-openshift-default-dockercfg-km7pb\""
Oct 30 19:18:39.610166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.608235    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"]
Oct 30 19:18:39.680743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.680704    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-istio-token\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.681001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.680763    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-socket\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.681001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.680870    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-envoy\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.681119 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.681011    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-data\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.681156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.681115    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-credential-socket\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.681211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.681177    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-certs\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.681290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.681273    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c927693-f0c9-4831-930e-9db28fd12ace-istiod-ca-cert\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.681347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.681328    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c927693-f0c9-4831-930e-9db28fd12ace-istio-podinfo\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.681419 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.681403    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t47fr\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-kube-api-access-t47fr\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782212 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782124    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-credential-socket\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782212 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782202    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-certs\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782252    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c927693-f0c9-4831-930e-9db28fd12ace-istiod-ca-cert\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782289    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c927693-f0c9-4831-930e-9db28fd12ace-istio-podinfo\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782320    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47fr\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-kube-api-access-t47fr\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782382    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-istio-token\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782436    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-socket\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782473    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-envoy\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782545    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-data\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.782758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782591    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-credential-socket\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.783181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.782881    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-certs\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.783181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.783099    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-data\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.783286 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.783202    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-socket\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.783753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.783727    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c927693-f0c9-4831-930e-9db28fd12ace-istiod-ca-cert\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.785387 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.785360    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c927693-f0c9-4831-930e-9db28fd12ace-istio-podinfo\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.785645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.785624    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-envoy\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.842229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.842198    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-istio-token\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:39.842644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:39.842619    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t47fr\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-kube-api-access-t47fr\") pod \"e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") " pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:40.135782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:40.135748    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:40.566602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:40.566520    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"]
Oct 30 19:18:40.570598 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:18:40.570560    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c927693_f0c9_4831_930e_9db28fd12ace.slice/crio-27ce1dd104496ec10662f651010c51b5f71550f5854f43212ea76b019015e7b4 WatchSource:0}: Error finding container 27ce1dd104496ec10662f651010c51b5f71550f5854f43212ea76b019015e7b4: Status 404 returned error can't find the container with id 27ce1dd104496ec10662f651010c51b5f71550f5854f43212ea76b019015e7b4
Oct 30 19:18:40.573230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:40.573196    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:40.573305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:40.573267    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:40.573305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:40.573299    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:40.750403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:40.750366    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" event={"ID":"1c927693-f0c9-4831-930e-9db28fd12ace","Type":"ContainerStarted","Data":"d71cdeb6d7b0be1032696a6c0467683c9ff557e35f0072bb78e64acdbc3f5c5d"}
Oct 30 19:18:40.750575 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:40.750424    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" event={"ID":"1c927693-f0c9-4831-930e-9db28fd12ace","Type":"ContainerStarted","Data":"27ce1dd104496ec10662f651010c51b5f71550f5854f43212ea76b019015e7b4"}
Oct 30 19:18:40.868709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:40.868652    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" podStartSLOduration=1.868637305 podStartE2EDuration="1.868637305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:40.86707063 +0000 UTC m=+1913.451553630" watchObservedRunningTime="2026-10-30 19:18:40.868637305 +0000 UTC m=+1913.453120302"
Oct 30 19:18:41.136689 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:41.136608    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:41.142415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:41.142384    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:41.757709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:41.757675    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:41.758745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:41.758727    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:18:46.188405 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:18:46.188368    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:18:47.291440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:47.291403    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"]
Oct 30 19:18:47.291856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:47.291654    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy" containerID="cri-o://86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb" gracePeriod=30
Oct 30 19:18:48.293066 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:48.288692    5232 scope.go:122] "RemoveContainer" containerID="c94add83c5d153a5bf23d8665b714992fd9f36b95960fb989ba83ca02d2a66d1"
Oct 30 19:18:48.918125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:48.918084    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"]
Oct 30 19:18:48.918386 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:48.918353    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7" podUID="5011494a-45e2-49d1-8a53-404a1b54f7d4" containerName="maas-api" containerID="cri-o://043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2" gracePeriod=30
Oct 30 19:18:49.267794 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.267770    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:18:49.330399 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.330357    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5011494a-45e2-49d1-8a53-404a1b54f7d4-kube-api-access-zcbrz\" (UniqueName: \"kubernetes.io/projected/5011494a-45e2-49d1-8a53-404a1b54f7d4-kube-api-access-zcbrz\") pod \"5011494a-45e2-49d1-8a53-404a1b54f7d4\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") "
Oct 30 19:18:49.330954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.330442    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls\") pod \"5011494a-45e2-49d1-8a53-404a1b54f7d4\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") "
Oct 30 19:18:49.330954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.330607    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5011494a-45e2-49d1-8a53-404a1b54f7d4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5011494a-45e2-49d1-8a53-404a1b54f7d4-openshift-service-ca\") pod \"5011494a-45e2-49d1-8a53-404a1b54f7d4\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") "
Oct 30 19:18:49.330954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.330669    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-metrics-certs\") pod \"5011494a-45e2-49d1-8a53-404a1b54f7d4\" (UID: \"5011494a-45e2-49d1-8a53-404a1b54f7d4\") "
Oct 30 19:18:49.331107 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.331009    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5011494a-45e2-49d1-8a53-404a1b54f7d4-openshift-service-ca" pod "5011494a-45e2-49d1-8a53-404a1b54f7d4" (UID: "5011494a-45e2-49d1-8a53-404a1b54f7d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:49.331244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.331217    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5011494a-45e2-49d1-8a53-404a1b54f7d4-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:49.332692 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.332666    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls" pod "5011494a-45e2-49d1-8a53-404a1b54f7d4" (UID: "5011494a-45e2-49d1-8a53-404a1b54f7d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:49.332692 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.332679    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-metrics-certs" pod "5011494a-45e2-49d1-8a53-404a1b54f7d4" (UID: "5011494a-45e2-49d1-8a53-404a1b54f7d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:49.332837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.332695    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5011494a-45e2-49d1-8a53-404a1b54f7d4-kube-api-access-zcbrz" pod "5011494a-45e2-49d1-8a53-404a1b54f7d4" (UID: "5011494a-45e2-49d1-8a53-404a1b54f7d4"). InnerVolumeSpecName "kube-api-access-zcbrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:49.432990 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.432952    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:49.433152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.432993    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcbrz\" (UniqueName: \"kubernetes.io/projected/5011494a-45e2-49d1-8a53-404a1b54f7d4-kube-api-access-zcbrz\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:49.433152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.433010    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5011494a-45e2-49d1-8a53-404a1b54f7d4-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:49.636591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.636558    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"]
Oct 30 19:18:49.637143 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637126    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:49.637143 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637144    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:49.637266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637157    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5011494a-45e2-49d1-8a53-404a1b54f7d4" containerName="maas-api"
Oct 30 19:18:49.637266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637163    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5011494a-45e2-49d1-8a53-404a1b54f7d4" containerName="maas-api"
Oct 30 19:18:49.637266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637181    5232 cpu_manager.go:414] "containerMap: removing container" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:49.637266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637187    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:49.637266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637261    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f3533ee-bc99-445c-9d65-62a1f00b3a85" containerName="cleanup"
Oct 30 19:18:49.637416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637269    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5011494a-45e2-49d1-8a53-404a1b54f7d4" containerName="maas-api"
Oct 30 19:18:49.637416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637282    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="25cd8f03-12af-4ebd-b751-278123bb4390" containerName="cleanup"
Oct 30 19:18:49.637416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.637288    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c25f605a-89a5-4b5b-b3ed-348bd91e541c" containerName="cleanup"
Oct 30 19:18:49.640557 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.640535    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.704099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.704062    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"]
Oct 30 19:18:49.705333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.705304    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-fb4a1b-openshift-default-dockercfg-8c7n6\""
Oct 30 19:18:49.737048 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737016    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-envoy\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.737240 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737141    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-credential-socket\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.737240 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737196    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-certs\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.737362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737244    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-socket\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.737417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737377    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-podinfo\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.737479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737454    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-data\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.737543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737489    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6hqd\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-kube-api-access-z6hqd\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.737543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737516    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-token\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.737643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.737622    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18349e05-80cb-4f86-abf7-84bb969e7f50-istiod-ca-cert\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.830062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.829969    5232 generic.go:356] "Generic (PLEG): container finished" podID="5011494a-45e2-49d1-8a53-404a1b54f7d4" containerID="043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2" exitCode=0
Oct 30 19:18:49.830062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.830024    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"
Oct 30 19:18:49.830062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.830045    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7" event={"ID":"5011494a-45e2-49d1-8a53-404a1b54f7d4","Type":"ContainerDied","Data":"043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2"}
Oct 30 19:18:49.830290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.830134    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7" event={"ID":"5011494a-45e2-49d1-8a53-404a1b54f7d4","Type":"ContainerDied","Data":"bd64d2e49fb3b1db6891fe46a69b1b2a6554d313846858468668ec467d6fa558"}
Oct 30 19:18:49.830290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.830162    5232 scope.go:122] "RemoveContainer" containerID="043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2"
Oct 30 19:18:49.839358 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839322    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18349e05-80cb-4f86-abf7-84bb969e7f50-istiod-ca-cert\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839504 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839381    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-envoy\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839504 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839441    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-credential-socket\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839504 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839468    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-certs\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839510    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-socket\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839585    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-podinfo\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839666    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-data\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839713    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hqd\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-kube-api-access-z6hqd\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839752    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-token\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839775    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-certs\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.839896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.839854    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-credential-socket\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.840131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.840005    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-socket\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.840131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.840090    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18349e05-80cb-4f86-abf7-84bb969e7f50-istiod-ca-cert\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.840312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.840163    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-data\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.840312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.840280    5232 scope.go:122] "RemoveContainer" containerID="043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2"
Oct 30 19:18:49.840627 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:18:49.840605    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2\": container with ID starting with 043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2 not found: ID does not exist" containerID="043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2"
Oct 30 19:18:49.840696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.840635    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2"} err="failed to get container status \"043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2\": rpc error: code = NotFound desc = could not find container \"043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2\": container with ID starting with 043f4a1f2fe0ec7cc19d98244856e29c4ad7982231c48666e33376e7d71ac1f2 not found: ID does not exist"
Oct 30 19:18:49.841699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.841678    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-envoy\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.842216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.842170    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-podinfo\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.913872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.913752    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-token\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.937867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.931209    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6hqd\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-kube-api-access-z6hqd\") pod \"e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") " pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.954676 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.954636    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:49.999728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:49.999683    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"]
Oct 30 19:18:50.013401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.013362    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f44636-74cf6cbfff-vjlr7"]
Oct 30 19:18:50.405903 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.405868    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"]
Oct 30 19:18:50.411112 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:18:50.411082    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18349e05_80cb_4f86_abf7_84bb969e7f50.slice/crio-a71324adad682e391efe0d7cfcb9ac07a8ef7f9926978c41216db3ede945e0ca WatchSource:0}: Error finding container a71324adad682e391efe0d7cfcb9ac07a8ef7f9926978c41216db3ede945e0ca: Status 404 returned error can't find the container with id a71324adad682e391efe0d7cfcb9ac07a8ef7f9926978c41216db3ede945e0ca
Oct 30 19:18:50.414525 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.414479    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:50.414632 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.414575    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:50.414632 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.414620    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:50.839859 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.839826    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" event={"ID":"18349e05-80cb-4f86-abf7-84bb969e7f50","Type":"ContainerStarted","Data":"ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040"}
Oct 30 19:18:50.840090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.839890    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" event={"ID":"18349e05-80cb-4f86-abf7-84bb969e7f50","Type":"ContainerStarted","Data":"a71324adad682e391efe0d7cfcb9ac07a8ef7f9926978c41216db3ede945e0ca"}
Oct 30 19:18:50.956225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.955136    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:50.957518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:50.957475    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" podStartSLOduration=1.957457797 podStartE2EDuration="1.957457797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:50.955566525 +0000 UTC m=+1923.540049523" watchObservedRunningTime="2026-10-30 19:18:50.957457797 +0000 UTC m=+1923.541940798"
Oct 30 19:18:51.542045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:51.542011    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5011494a-45e2-49d1-8a53-404a1b54f7d4" path="/var/lib/kubelet/pods/5011494a-45e2-49d1-8a53-404a1b54f7d4/volumes"
Oct 30 19:18:51.960975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:51.960939    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:52.596953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.596915    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:18:52.676045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676012    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-kube-api-access-qkx6v\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-kube-api-access-qkx6v\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676057    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-certs\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676097    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-socket\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676185    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-envoy\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676223    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-credential-socket\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676306    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-podinfo\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676442 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676353    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-token\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-token\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676502 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676434    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-certs" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:52.676502 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676468    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istiod-ca-cert\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676502    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-data\") pod \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\" (UID: \"cff4f45b-6dbb-41e5-9a75-59aa00b8712e\") "
Oct 30 19:18:52.676628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676561    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-socket" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:52.676888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676862    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-data" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:52.677001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676902    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istiod-ca-cert" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:52.677001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.676960    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-credential-socket" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:52.677174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.677153    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.677241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.677179    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.677241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.677197    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.677241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.677210    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.677241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.677224    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.678706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.678680    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-podinfo" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:52.678833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.678751    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-token" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:52.678923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.678899    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-kube-api-access-qkx6v" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "kube-api-access-qkx6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:52.678965 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.678925    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-envoy" pod "cff4f45b-6dbb-41e5-9a75-59aa00b8712e" (UID: "cff4f45b-6dbb-41e5-9a75-59aa00b8712e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:52.778851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.778802    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkx6v\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-kube-api-access-qkx6v\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.778851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.778851    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.779045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.778861    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.779045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.778870    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff4f45b-6dbb-41e5-9a75-59aa00b8712e-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:52.853587 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.853550    5232 generic.go:356] "Generic (PLEG): container finished" podID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerID="86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb" exitCode=0
Oct 30 19:18:52.853799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.853607    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"
Oct 30 19:18:52.853799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.853635    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" event={"ID":"cff4f45b-6dbb-41e5-9a75-59aa00b8712e","Type":"ContainerDied","Data":"86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb"}
Oct 30 19:18:52.853799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.853696    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk" event={"ID":"cff4f45b-6dbb-41e5-9a75-59aa00b8712e","Type":"ContainerDied","Data":"71f0f62f1bc8385ad1702d185a553fc91f091327c8f281381b0aa225c2626845"}
Oct 30 19:18:52.853799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.853726    5232 scope.go:122] "RemoveContainer" containerID="86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb"
Oct 30 19:18:52.854265 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.854247    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:52.855435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.855417    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:18:52.864828 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.864795    5232 scope.go:122] "RemoveContainer" containerID="fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3"
Oct 30 19:18:52.875042 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.875020    5232 scope.go:122] "RemoveContainer" containerID="86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb"
Oct 30 19:18:52.875342 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:18:52.875323    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb\": container with ID starting with 86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb not found: ID does not exist" containerID="86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb"
Oct 30 19:18:52.875391 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.875353    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb"} err="failed to get container status \"86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb\": rpc error: code = NotFound desc = could not find container \"86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb\": container with ID starting with 86c11cbbdd466af55586d8de60e8b23804ba0e48db496522537bc4e41680daeb not found: ID does not exist"
Oct 30 19:18:52.875391 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.875373    5232 scope.go:122] "RemoveContainer" containerID="fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3"
Oct 30 19:18:52.875621 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:18:52.875608    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3\": container with ID starting with fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3 not found: ID does not exist" containerID="fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3"
Oct 30 19:18:52.875655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:52.875625    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3"} err="failed to get container status \"fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3\": rpc error: code = NotFound desc = could not find container \"fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3\": container with ID starting with fcfcd9a363159afb469e9c8d2d16a15213d2746003b4405ccb26fc431b5620f3 not found: ID does not exist"
Oct 30 19:18:53.131382 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:53.131290    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"]
Oct 30 19:18:53.138974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:53.138934    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0d2ef6-openshift-default-84c4b65565-q8tkk"]
Oct 30 19:18:53.540175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:18:53.540096    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" path="/var/lib/kubelet/pods/cff4f45b-6dbb-41e5-9a75-59aa00b8712e/volumes"
Oct 30 19:19:06.250384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.250338    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"]
Oct 30 19:19:06.251053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.251033    5232 cpu_manager.go:414] "containerMap: removing container" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy"
Oct 30 19:19:06.251139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.251054    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy"
Oct 30 19:19:06.251193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.251183    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy"
Oct 30 19:19:06.251228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.251193    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy"
Oct 30 19:19:06.254901 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.254879    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.297573 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.297538    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"]
Oct 30 19:19:06.298073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.298060    5232 cpu_manager.go:414] "containerMap: removing container" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy"
Oct 30 19:19:06.298116 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.298074    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cff4f45b-6dbb-41e5-9a75-59aa00b8712e" containerName="istio-proxy"
Oct 30 19:19:06.301466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.301440    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.345467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.345427    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"]
Oct 30 19:19:06.357725 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.357678    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"]
Oct 30 19:19:06.361462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361425    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.361613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361499    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dbx4\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-kube-api-access-2dbx4\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.361613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361535    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.361613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361563    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.361613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361590    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.361847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361637    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33949089-2503-4339-b075-87a671d3b83e-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.361847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361713    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.361847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361750    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d655307a-d38e-4ecd-99dd-8d7b24721114-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.361847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361776    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.361847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361804    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.362115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361867    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.362115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361904    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33949089-2503-4339-b075-87a671d3b83e-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.362115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361935    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.362115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.361958    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.362115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.362006    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpzh9\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-kube-api-access-tpzh9\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.362115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.362089    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.362415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.362119    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.362415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.362177    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.463583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463511    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.463583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463586    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d655307a-d38e-4ecd-99dd-8d7b24721114-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.463583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463615    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.464046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463649    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.464046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463721    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.464046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463759    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33949089-2503-4339-b075-87a671d3b83e-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.464046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463787    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.464046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463826    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.464046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463879    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzh9\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-kube-api-access-tpzh9\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.464046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.463974    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.464046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.464005    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.464415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.464235    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.464415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.464329    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.464415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.464357    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dbx4\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-kube-api-access-2dbx4\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.464415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.464386    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.464591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.464413    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.464591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.464443    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.464591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.464499    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33949089-2503-4339-b075-87a671d3b83e-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.468872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.465614    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33949089-2503-4339-b075-87a671d3b83e-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.468872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.466710    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.468872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.467408    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d655307a-d38e-4ecd-99dd-8d7b24721114-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.468872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.468097    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.474005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.473967    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.474247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.474219    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.474330 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.474247    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.474402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.474356    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.474492 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.474470    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.474615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.474596    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.475254 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.475223    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.475354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.475298    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.476639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.476614    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33949089-2503-4339-b075-87a671d3b83e-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.477014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.476966    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.525215 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.525138    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpzh9\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-kube-api-access-tpzh9\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.528518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.528489    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:06.529031 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.529012    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.529266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.529242    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dbx4\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-kube-api-access-2dbx4\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.568858 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.568799    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:06.616143 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:06.616111    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:07.002085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.002048    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"]
Oct 30 19:19:07.005821 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:19:07.005778    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd655307a_d38e_4ecd_99dd_8d7b24721114.slice/crio-f0f892d30c80455505b0fad9d5d6ab5b102a8cbd2101f15c9594ee995ae351bf WatchSource:0}: Error finding container f0f892d30c80455505b0fad9d5d6ab5b102a8cbd2101f15c9594ee995ae351bf: Status 404 returned error can't find the container with id f0f892d30c80455505b0fad9d5d6ab5b102a8cbd2101f15c9594ee995ae351bf
Oct 30 19:19:07.008615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.008577    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:07.008694 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.008651    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:07.008694 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.008683    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:07.055996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.055298    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"]
Oct 30 19:19:07.059673 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:19:07.059639    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33949089_2503_4339_b075_87a671d3b83e.slice/crio-f4f5ea03cb2a79ba52136f61eac4a1aeff961545446096bbdb78369aa264b383 WatchSource:0}: Error finding container f4f5ea03cb2a79ba52136f61eac4a1aeff961545446096bbdb78369aa264b383: Status 404 returned error can't find the container with id f4f5ea03cb2a79ba52136f61eac4a1aeff961545446096bbdb78369aa264b383
Oct 30 19:19:07.062307 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.062272    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:07.062396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.062340    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:07.062396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.062375    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:07.989207 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.989168    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" event={"ID":"d655307a-d38e-4ecd-99dd-8d7b24721114","Type":"ContainerStarted","Data":"da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64"}
Oct 30 19:19:07.989749 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.989731    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" event={"ID":"d655307a-d38e-4ecd-99dd-8d7b24721114","Type":"ContainerStarted","Data":"f0f892d30c80455505b0fad9d5d6ab5b102a8cbd2101f15c9594ee995ae351bf"}
Oct 30 19:19:07.993079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.993054    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" event={"ID":"33949089-2503-4339-b075-87a671d3b83e","Type":"ContainerStarted","Data":"be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f"}
Oct 30 19:19:07.993269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:07.993255    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" event={"ID":"33949089-2503-4339-b075-87a671d3b83e","Type":"ContainerStarted","Data":"f4f5ea03cb2a79ba52136f61eac4a1aeff961545446096bbdb78369aa264b383"}
Oct 30 19:19:08.219005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:08.218949    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podStartSLOduration=2.218935282 podStartE2EDuration="2.218935282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:08.166715234 +0000 UTC m=+1940.751198232" watchObservedRunningTime="2026-10-30 19:19:08.218935282 +0000 UTC m=+1940.803418346"
Oct 30 19:19:08.302874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:08.302721    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podStartSLOduration=2.302701401 podStartE2EDuration="2.302701401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:08.295350697 +0000 UTC m=+1940.879833694" watchObservedRunningTime="2026-10-30 19:19:08.302701401 +0000 UTC m=+1940.887184401"
Oct 30 19:19:08.584894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:08.571349    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:08.584894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:08.571512    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:19:08.584894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:08.571565    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Oct 30 19:19:08.616480 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:08.616447    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:08.617696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:08.617665    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:19:08.617759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:08.617725    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Oct 30 19:19:09.569836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:09.569710    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:19:09.569836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:09.569773    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Oct 30 19:19:09.616832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:09.616782    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:19:09.617001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:09.616860    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Oct 30 19:19:10.574267 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:10.574225    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:19:10.575629 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:10.575592    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Oct 30 19:19:10.617033 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:10.616995    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:19:10.617224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:10.617068    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Oct 30 19:19:11.571296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:11.571257    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:19:11.571591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:11.571563    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Oct 30 19:19:11.616909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:11.616878    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:19:11.617379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:11.617354    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Oct 30 19:19:12.582262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:12.582224    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:19:12.582535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:12.582502    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Oct 30 19:19:12.617138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:12.617099    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:19:12.617626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:12.617311    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Oct 30 19:19:13.569983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:13.569784    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:19:13.569983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:13.569864    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Oct 30 19:19:13.616845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:13.616612    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:19:13.616845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:13.616672    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.115:15021/healthz/ready\": dial tcp 10.133.0.115:15021: connect: connection refused"
Oct 30 19:19:14.574466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:14.574434    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:14.575019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:14.574840    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:14.575584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:14.575569    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:19:14.621066 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:14.621037    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:14.621326 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:14.621306    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:14.621962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:14.621945    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:19:16.188582 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:19:16.188544    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:19:21.411972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.411942    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"]
Oct 30 19:19:21.416327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.416305    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.517493 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.517454    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.517691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.517600    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.517775 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.517702    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.517775 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.517758    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7ca1784-2282-4c82-9c70-df58b417e61f-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.517941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.517777    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b748g\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-kube-api-access-b748g\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.517941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.517796    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.517941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.517852    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.517941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.517910    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.518162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.518059    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.519655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.519623    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"]
Oct 30 19:19:21.619558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.619521    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.619753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.619594    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.619852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.619748    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.619987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.619961    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620097    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620105    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620210 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620126    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620210 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620174    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620210 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620195    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7ca1784-2282-4c82-9c70-df58b417e61f-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620342 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620216    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b748g\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-kube-api-access-b748g\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620342 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620237    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620342 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620282    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620386    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.620931 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.620908    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7ca1784-2282-4c82-9c70-df58b417e61f-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.622203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.622181    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.622688 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.622672    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.679166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.679091    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.679166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.679127    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b748g\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-kube-api-access-b748g\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:21.730877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:21.730844    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:22.171636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:22.171600    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"]
Oct 30 19:19:22.174661 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:19:22.174626    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ca1784_2282_4c82_9c70_df58b417e61f.slice/crio-aa38636dc45ea1954cdaf8908371567ed0faec48e27df38ea11dffd8459a72b8 WatchSource:0}: Error finding container aa38636dc45ea1954cdaf8908371567ed0faec48e27df38ea11dffd8459a72b8: Status 404 returned error can't find the container with id aa38636dc45ea1954cdaf8908371567ed0faec48e27df38ea11dffd8459a72b8
Oct 30 19:19:22.177498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:22.177462    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:22.177593 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:22.177538    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:22.177593 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:22.177568    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:23.179261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:23.179212    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" event={"ID":"c7ca1784-2282-4c82-9c70-df58b417e61f","Type":"ContainerStarted","Data":"d86d0272f354e9bcb8e1b1d2f8198a199ef2c88a12a065aabfef65f9417e12aa"}
Oct 30 19:19:23.179670 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:23.179315    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" event={"ID":"c7ca1784-2282-4c82-9c70-df58b417e61f","Type":"ContainerStarted","Data":"aa38636dc45ea1954cdaf8908371567ed0faec48e27df38ea11dffd8459a72b8"}
Oct 30 19:19:23.397161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:23.397107    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podStartSLOduration=2.397094675 podStartE2EDuration="2.397094675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:23.34932733 +0000 UTC m=+1955.933810329" watchObservedRunningTime="2026-10-30 19:19:23.397094675 +0000 UTC m=+1955.981577669"
Oct 30 19:19:23.731701 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:23.731666    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:23.733241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:23.733212    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:19:23.733379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:23.733263    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 19:19:24.731337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:24.731300    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:19:24.731881 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:24.731361    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 19:19:25.731375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:25.731339    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:19:25.731804 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:25.731400    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 19:19:26.731650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:26.731613    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:19:26.732127 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:26.731686    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 19:19:27.731679 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:27.731638    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:19:27.732222 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:27.731692    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 19:19:28.732034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:28.731990    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:19:28.732575 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:28.732045    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 19:19:29.736289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:29.736259    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:29.736686 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:29.736522    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:29.737446 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:29.737423    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:19:35.935832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:35.935779    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"]
Oct 30 19:19:35.940390 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:35.940365    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:35.982170 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:35.982132    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:35.982335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:35.982237    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8327232a-4d38-4266-a5e8-dc464ea7fb0a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:35.982400 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:35.982340    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-metrics-certs\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:35.982400 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:35.982384    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvl8n\" (UniqueName: \"kubernetes.io/projected/8327232a-4d38-4266-a5e8-dc464ea7fb0a-kube-api-access-pvl8n\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:35.990707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:35.990682    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-4ec347-serving-cert\""
Oct 30 19:19:36.043478 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.043438    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"]
Oct 30 19:19:36.083569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.083536    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.083830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.083606    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8327232a-4d38-4266-a5e8-dc464ea7fb0a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.083830 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:19:36.083731    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-serving-cert: secret "maas-api-e2e-ipp-a-4ec347-serving-cert" not found
Oct 30 19:19:36.083830 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:19:36.083807    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls podName:8327232a-4d38-4266-a5e8-dc464ea7fb0a nodeName:}" failed. No retries permitted until 2026-10-30 19:19:36.583785533 +0000 UTC m=+1969.168268524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls") pod "maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl" (UID: "8327232a-4d38-4266-a5e8-dc464ea7fb0a") : secret "maas-api-e2e-ipp-a-4ec347-serving-cert" not found
Oct 30 19:19:36.084059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.083845    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-metrics-certs\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.084059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.083937    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvl8n\" (UniqueName: \"kubernetes.io/projected/8327232a-4d38-4266-a5e8-dc464ea7fb0a-kube-api-access-pvl8n\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.084489 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.084456    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8327232a-4d38-4266-a5e8-dc464ea7fb0a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.087083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.087051    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-metrics-certs\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.145346 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.145312    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvl8n\" (UniqueName: \"kubernetes.io/projected/8327232a-4d38-4266-a5e8-dc464ea7fb0a-kube-api-access-pvl8n\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.594774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.594739    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.597208 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.597183    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:36.856792 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:36.856705    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:37.238769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:37.238649    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"]
Oct 30 19:19:37.241846 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:19:37.241805    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8327232a_4d38_4266_a5e8_dc464ea7fb0a.slice/crio-7634f972eb8c310ed8631bd3046a6b71f3ac934f0f6c3dcc4f1761a828740a1b WatchSource:0}: Error finding container 7634f972eb8c310ed8631bd3046a6b71f3ac934f0f6c3dcc4f1761a828740a1b: Status 404 returned error can't find the container with id 7634f972eb8c310ed8631bd3046a6b71f3ac934f0f6c3dcc4f1761a828740a1b
Oct 30 19:19:37.243312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:37.243288    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:19:37.280178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:37.280145    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl" event={"ID":"8327232a-4d38-4266-a5e8-dc464ea7fb0a","Type":"ContainerStarted","Data":"7634f972eb8c310ed8631bd3046a6b71f3ac934f0f6c3dcc4f1761a828740a1b"}
Oct 30 19:19:38.288912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:38.288873    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl" event={"ID":"8327232a-4d38-4266-a5e8-dc464ea7fb0a","Type":"ContainerStarted","Data":"63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf"}
Oct 30 19:19:38.289333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:38.289004    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:38.404384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:38.404329    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl" podStartSLOduration=2.578349615 podStartE2EDuration="3.40431315s" totalImagesPullingTime="825.963535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:38.403042466 +0000 UTC m=+1970.987525465" watchObservedRunningTime="2026-10-30 19:19:38.40431315 +0000 UTC m=+1970.988796141"
Oct 30 19:19:44.301580 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:44.301551    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:19:46.188683 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:19:46.188644    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:19:53.417614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:53.417577    5232 generic.go:356] "Generic (PLEG): container finished" podID="1c927693-f0c9-4831-930e-9db28fd12ace" containerID="d71cdeb6d7b0be1032696a6c0467683c9ff557e35f0072bb78e64acdbc3f5c5d" exitCode=0
Oct 30 19:19:53.418182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:53.417652    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" event={"ID":"1c927693-f0c9-4831-930e-9db28fd12ace","Type":"ContainerDied","Data":"d71cdeb6d7b0be1032696a6c0467683c9ff557e35f0072bb78e64acdbc3f5c5d"}
Oct 30 19:19:53.418251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:53.418210    5232 scope.go:122] "RemoveContainer" containerID="d71cdeb6d7b0be1032696a6c0467683c9ff557e35f0072bb78e64acdbc3f5c5d"
Oct 30 19:19:53.419710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:53.419682    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:53.419777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:53.419761    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:53.419829 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:53.419792    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:54.425687 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:54.425654    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" event={"ID":"1c927693-f0c9-4831-930e-9db28fd12ace","Type":"ContainerStarted","Data":"ab50489bfac205d23356f178f0383b56a1d4c34cb33f9cb038a4f7faadd5b075"}
Oct 30 19:19:55.136517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:55.136477    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:19:55.162856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:55.162078    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:19:55.162856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:55.162177    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:19:56.136643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:56.136605    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:19:56.137196 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:56.136676    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:19:56.762854 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:56.761948    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:19:57.136506 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:57.136466    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:19:57.136716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:57.136541    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:19:58.136246 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.136214    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:19:58.136453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.136351    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:19:58.398444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.398359    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"]
Oct 30 19:19:58.399194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.399163    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy" containerID="cri-o://d86d0272f354e9bcb8e1b1d2f8198a199ef2c88a12a065aabfef65f9417e12aa" gracePeriod=30
Oct 30 19:19:58.465945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.465911    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"]
Oct 30 19:19:58.466361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.466337    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy" containerID="cri-o://be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f" gracePeriod=30
Oct 30 19:19:58.470698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.470670    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"]
Oct 30 19:19:58.471180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.471154    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy" containerID="cri-o://da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64" gracePeriod=30
Oct 30 19:19:58.472952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.472933    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"]
Oct 30 19:19:58.473121 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:58.473100    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy" containerID="cri-o://d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12" gracePeriod=30
Oct 30 19:19:59.160835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:59.136777    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:19:59.160835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:59.136873    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:19:59.577584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:59.577492    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:19:59.578010 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:59.577559    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:19:59.624156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:59.624105    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:19:59.624354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:59.624295    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:19:59.739600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:59.739556    5232 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:19:59.739768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:19:59.739616    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:20:00.141867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:00.141832    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:20:00.143944 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:00.143920    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:20:03.581345 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.581316    5232 generic.go:356] "Generic (PLEG): container finished" podID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerID="d86d0272f354e9bcb8e1b1d2f8198a199ef2c88a12a065aabfef65f9417e12aa" exitCode=0
Oct 30 19:20:03.581976 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.581606    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" event={"ID":"c7ca1784-2282-4c82-9c70-df58b417e61f","Type":"ContainerDied","Data":"d86d0272f354e9bcb8e1b1d2f8198a199ef2c88a12a065aabfef65f9417e12aa"}
Oct 30 19:20:03.808497 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.808460    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:20:03.895941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.895898    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:20:03.901179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901139    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-envoy\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.901362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901238    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-certs\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.901362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901302    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-kube-api-access-b748g\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-kube-api-access-b748g\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.901362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901350    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-token\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-token\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.901570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901410    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-credential-socket\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.901570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901473    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-podinfo\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.901570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901507    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7ca1784-2282-4c82-9c70-df58b417e61f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7ca1784-2282-4c82-9c70-df58b417e61f-istiod-ca-cert\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.901751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901657    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-socket\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.901751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.901699    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-data\") pod \"c7ca1784-2282-4c82-9c70-df58b417e61f\" (UID: \"c7ca1784-2282-4c82-9c70-df58b417e61f\") "
Oct 30 19:20:03.902509 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.902484    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-data" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.904294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.904264    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-certs" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.904468 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.904448    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-socket" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.904632 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.904612    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7ca1784-2282-4c82-9c70-df58b417e61f-istiod-ca-cert" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:03.904696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.904664    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-credential-socket" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.907961 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.907885    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-token" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:03.909935 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.909908    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-kube-api-access-b748g" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "kube-api-access-b748g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:03.911103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.911062    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-podinfo" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:20:03.915824 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.915785    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-envoy" pod "c7ca1784-2282-4c82-9c70-df58b417e61f" (UID: "c7ca1784-2282-4c82-9c70-df58b417e61f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.917105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.917087    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:20:03.921068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:03.921041    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.002758    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istiod-ca-cert\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.002857    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-data\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.002959    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-certs\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003006    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-envoy\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003040    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-envoy\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003091    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-data\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003113    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-podinfo\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003171    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-podinfo\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003210    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-kube-api-access-qlqv4\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-kube-api-access-qlqv4\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003232    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-envoy\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003327    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-token\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-token\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003350    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-credential-socket\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003379    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-istio-token\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-istio-token\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003402    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-socket\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003430    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-token\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-token\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003455    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33949089-2503-4339-b075-87a671d3b83e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33949089-2503-4339-b075-87a671d3b83e-istiod-ca-cert\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003490    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-kube-api-access-tpzh9\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-kube-api-access-tpzh9\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003515    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d655307a-d38e-4ecd-99dd-8d7b24721114-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d655307a-d38e-4ecd-99dd-8d7b24721114-istiod-ca-cert\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003590    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-certs\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003638    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-socket\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003684    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-data\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003717    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-socket\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003769    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-credential-socket\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003793    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-certs\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003871    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33949089-2503-4339-b075-87a671d3b83e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33949089-2503-4339-b075-87a671d3b83e-istio-podinfo\") pod \"33949089-2503-4339-b075-87a671d3b83e\" (UID: \"33949089-2503-4339-b075-87a671d3b83e\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003934    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-credential-socket\") pod \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\" (UID: \"5fcbf443-86fd-4ea5-9c23-f6d7c957be18\") "
Oct 30 19:20:04.004008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.003983    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-kube-api-access-2dbx4\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-kube-api-access-2dbx4\") pod \"d655307a-d38e-4ecd-99dd-8d7b24721114\" (UID: \"d655307a-d38e-4ecd-99dd-8d7b24721114\") "
Oct 30 19:20:04.006185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006152    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-certs" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.006312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006230    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-data" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.006381 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006348    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006381    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006399    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b748g\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-kube-api-access-b748g\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006417    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006433    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006449    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006464    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7ca1784-2282-4c82-9c70-df58b417e61f-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006479    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006494    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ca1784-2282-4c82-9c70-df58b417e61f-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.006648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006517    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istiod-ca-cert" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:04.006967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.006842    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-data" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.007173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.007111    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-socket" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.007312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.007238    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-socket" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.009907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.008599    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-credential-socket" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.009907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.009638    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-credential-socket" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.012866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.010237    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33949089-2503-4339-b075-87a671d3b83e-istiod-ca-cert" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:04.012866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.010426    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d655307a-d38e-4ecd-99dd-8d7b24721114-istiod-ca-cert" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:04.012866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.010478    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-data" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.012866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.010546    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-podinfo" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:20:04.012866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.010733    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-istio-token" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:04.012866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.010804    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-socket" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.012866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.011046    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-certs" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.012866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.011220    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-credential-socket" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.013335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.012906    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-token" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:04.013335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.013228    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-certs" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.013335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.013313    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-kube-api-access-2dbx4" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "kube-api-access-2dbx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:04.014372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.014316    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-kube-api-access-tpzh9" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "kube-api-access-tpzh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:04.015204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.015167    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-podinfo" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:20:04.015204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.015196    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-envoy" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.015373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.015348    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-envoy" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.015500 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.015487    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-envoy" pod "d655307a-d38e-4ecd-99dd-8d7b24721114" (UID: "d655307a-d38e-4ecd-99dd-8d7b24721114"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:04.015653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.015634    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-kube-api-access-qlqv4" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "kube-api-access-qlqv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:04.015728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.015710    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33949089-2503-4339-b075-87a671d3b83e-istio-podinfo" pod "33949089-2503-4339-b075-87a671d3b83e" (UID: "33949089-2503-4339-b075-87a671d3b83e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:20:04.017137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.017108    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-token" pod "5fcbf443-86fd-4ea5-9c23-f6d7c957be18" (UID: "5fcbf443-86fd-4ea5-9c23-f6d7c957be18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.109941    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d655307a-d38e-4ecd-99dd-8d7b24721114-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.109971    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.109981    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.109990    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.109999    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110008    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110016    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110026    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33949089-2503-4339-b075-87a671d3b83e-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110034    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110043    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dbx4\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-kube-api-access-2dbx4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110054    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110062    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110070    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110078    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110085    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110092    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110100    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110108    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.110100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110119    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlqv4\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-kube-api-access-qlqv4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.111228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110127    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.111228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110137    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fcbf443-86fd-4ea5-9c23-f6d7c957be18-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.111228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110145    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33949089-2503-4339-b075-87a671d3b83e-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.111228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110154    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.111228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110163    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d655307a-d38e-4ecd-99dd-8d7b24721114-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.111228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110171    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d655307a-d38e-4ecd-99dd-8d7b24721114-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.111228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110715    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33949089-2503-4339-b075-87a671d3b83e-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.111228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.110727    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpzh9\" (UniqueName: \"kubernetes.io/projected/33949089-2503-4339-b075-87a671d3b83e-kube-api-access-tpzh9\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.592053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.591619    5232 generic.go:356] "Generic (PLEG): container finished" podID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerID="d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12" exitCode=0
Oct 30 19:20:04.592053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.591675    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"
Oct 30 19:20:04.592053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.591703    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" event={"ID":"5fcbf443-86fd-4ea5-9c23-f6d7c957be18","Type":"ContainerDied","Data":"d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12"}
Oct 30 19:20:04.592053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.591762    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6" event={"ID":"5fcbf443-86fd-4ea5-9c23-f6d7c957be18","Type":"ContainerDied","Data":"41e65387dfbf8894669071081c59d4190485e9ecbbb97d878df00f25282afb08"}
Oct 30 19:20:04.592053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.591797    5232 scope.go:122] "RemoveContainer" containerID="d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12"
Oct 30 19:20:04.593626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.593600    5232 generic.go:356] "Generic (PLEG): container finished" podID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerID="da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64" exitCode=0
Oct 30 19:20:04.593718 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.593705    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" event={"ID":"d655307a-d38e-4ecd-99dd-8d7b24721114","Type":"ContainerDied","Data":"da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64"}
Oct 30 19:20:04.593758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.593741    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5" event={"ID":"d655307a-d38e-4ecd-99dd-8d7b24721114","Type":"ContainerDied","Data":"f0f892d30c80455505b0fad9d5d6ab5b102a8cbd2101f15c9594ee995ae351bf"}
Oct 30 19:20:04.593799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.593789    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"
Oct 30 19:20:04.601148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.599768    5232 generic.go:356] "Generic (PLEG): container finished" podID="33949089-2503-4339-b075-87a671d3b83e" containerID="be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f" exitCode=0
Oct 30 19:20:04.601148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.599919    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"
Oct 30 19:20:04.601148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.600221    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" event={"ID":"33949089-2503-4339-b075-87a671d3b83e","Type":"ContainerDied","Data":"be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f"}
Oct 30 19:20:04.601148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.600325    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn" event={"ID":"33949089-2503-4339-b075-87a671d3b83e","Type":"ContainerDied","Data":"f4f5ea03cb2a79ba52136f61eac4a1aeff961545446096bbdb78369aa264b383"}
Oct 30 19:20:04.602659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.602627    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl" event={"ID":"c7ca1784-2282-4c82-9c70-df58b417e61f","Type":"ContainerDied","Data":"aa38636dc45ea1954cdaf8908371567ed0faec48e27df38ea11dffd8459a72b8"}
Oct 30 19:20:04.602756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.602741    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"
Oct 30 19:20:04.608126 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.607987    5232 scope.go:122] "RemoveContainer" containerID="06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104"
Oct 30 19:20:04.621609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.621587    5232 scope.go:122] "RemoveContainer" containerID="d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12"
Oct 30 19:20:04.622002 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:20:04.621979    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12\": container with ID starting with d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12 not found: ID does not exist" containerID="d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12"
Oct 30 19:20:04.622074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.622014    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12"} err="failed to get container status \"d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12\": rpc error: code = NotFound desc = could not find container \"d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12\": container with ID starting with d8fe15a0ac8a555ed53ec2294c908d14713871eec0fa10ccc1366e22b866bd12 not found: ID does not exist"
Oct 30 19:20:04.622074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.622036    5232 scope.go:122] "RemoveContainer" containerID="06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104"
Oct 30 19:20:04.622292 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:20:04.622272    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104\": container with ID starting with 06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104 not found: ID does not exist" containerID="06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104"
Oct 30 19:20:04.622343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.622299    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104"} err="failed to get container status \"06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104\": rpc error: code = NotFound desc = could not find container \"06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104\": container with ID starting with 06cd56ba22b6257c507b5dfd8233fd1c43287174ff72c6e89cc9cb575fb48104 not found: ID does not exist"
Oct 30 19:20:04.622343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.622315    5232 scope.go:122] "RemoveContainer" containerID="da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64"
Oct 30 19:20:04.633004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.632980    5232 scope.go:122] "RemoveContainer" containerID="da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64"
Oct 30 19:20:04.633341 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:20:04.633321    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64\": container with ID starting with da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64 not found: ID does not exist" containerID="da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64"
Oct 30 19:20:04.633397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.633360    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64"} err="failed to get container status \"da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64\": rpc error: code = NotFound desc = could not find container \"da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64\": container with ID starting with da8051c7d63079ec8793f2652a93208919881f27bad82371dbf9c6676d517b64 not found: ID does not exist"
Oct 30 19:20:04.633397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.633376    5232 scope.go:122] "RemoveContainer" containerID="be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f"
Oct 30 19:20:04.672328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.672301    5232 scope.go:122] "RemoveContainer" containerID="be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f"
Oct 30 19:20:04.672756 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:20:04.672724    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f\": container with ID starting with be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f not found: ID does not exist" containerID="be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f"
Oct 30 19:20:04.672906 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.672767    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f"} err="failed to get container status \"be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f\": rpc error: code = NotFound desc = could not find container \"be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f\": container with ID starting with be012798bb71180477b125ba9285b10d92f075865ae9613cc29c16806b0e330f not found: ID does not exist"
Oct 30 19:20:04.672906 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.672795    5232 scope.go:122] "RemoveContainer" containerID="d86d0272f354e9bcb8e1b1d2f8198a199ef2c88a12a065aabfef65f9417e12aa"
Oct 30 19:20:04.774397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.772900    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"]
Oct 30 19:20:04.776776 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.776693    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wktbl"]
Oct 30 19:20:04.885090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.885049    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"]
Oct 30 19:20:04.888444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.888412    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-ndrt6"]
Oct 30 19:20:04.992206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.992119    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"]
Oct 30 19:20:04.995302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:04.995270    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-2bhpn"]
Oct 30 19:20:05.101322 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:05.101279    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"]
Oct 30 19:20:05.107525 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:05.107489    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-wccr5"]
Oct 30 19:20:05.564710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:05.564675    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33949089-2503-4339-b075-87a671d3b83e" path="/var/lib/kubelet/pods/33949089-2503-4339-b075-87a671d3b83e/volumes"
Oct 30 19:20:05.565263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:05.565246    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" path="/var/lib/kubelet/pods/5fcbf443-86fd-4ea5-9c23-f6d7c957be18/volumes"
Oct 30 19:20:05.565765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:05.565747    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" path="/var/lib/kubelet/pods/c7ca1784-2282-4c82-9c70-df58b417e61f/volumes"
Oct 30 19:20:05.566668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:05.566649    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" path="/var/lib/kubelet/pods/d655307a-d38e-4ecd-99dd-8d7b24721114/volumes"
Oct 30 19:20:08.854772 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:08.854727    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:20:08.855383 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:08.854808    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:20:10.651332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:10.651249    5232 generic.go:356] "Generic (PLEG): container finished" podID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerID="ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040" exitCode=0
Oct 30 19:20:10.651745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:10.651320    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" event={"ID":"18349e05-80cb-4f86-abf7-84bb969e7f50","Type":"ContainerDied","Data":"ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040"}
Oct 30 19:20:10.651787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:10.651767    5232 scope.go:122] "RemoveContainer" containerID="ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040"
Oct 30 19:20:10.653397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:10.653366    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:10.653483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:10.653437    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:10.653524 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:10.653484    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:11.666408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:11.666372    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" event={"ID":"18349e05-80cb-4f86-abf7-84bb969e7f50","Type":"ContainerStarted","Data":"3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8"}
Oct 30 19:20:11.957894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:11.956086    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:20:11.957894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:11.956214    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:20:11.957894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:11.956256    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 19:20:12.957268 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:12.957228    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:20:12.957958 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:12.957911    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 19:20:13.956836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:13.956190    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:20:13.956836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:13.956251    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 19:20:14.958207 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:14.958166    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:20:14.958873 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:14.958801    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 19:20:15.962456 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:15.962412    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:20:15.963110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:15.962492    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 19:20:16.191027 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:20:16.190982    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:20:16.959443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:16.959411    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:20:16.959782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:16.959762    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:20:16.960388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:16.960373    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:20:32.733373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.733339    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"]
Oct 30 19:20:32.734856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734831    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy"
Oct 30 19:20:32.734856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734853    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734866    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734871    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734886    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734891    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734915    5232 cpu_manager.go:414] "containerMap: removing container" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734920    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734931    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.734935    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy"
Oct 30 19:20:32.735020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.735017    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d655307a-d38e-4ecd-99dd-8d7b24721114" containerName="istio-proxy"
Oct 30 19:20:32.735305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.735027    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7ca1784-2282-4c82-9c70-df58b417e61f" containerName="istio-proxy"
Oct 30 19:20:32.735305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.735036    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy"
Oct 30 19:20:32.735305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.735045    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="33949089-2503-4339-b075-87a671d3b83e" containerName="istio-proxy"
Oct 30 19:20:32.738202 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.738177    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.805274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.805229    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-65d741-openshift-default-dockercfg-lf49w\""
Oct 30 19:20:32.835718 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.835673    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-podinfo\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.836065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.836007    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-credential-socket\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.836211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.836085    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-data\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.836211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.836119    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/372dccb4-44bc-44c8-8d0b-819378f3f085-istiod-ca-cert\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.836327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.836225    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29jkz\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-kube-api-access-29jkz\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.836327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.836284    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-socket\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.836327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.836307    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-certs\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.836484 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.836333    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-token\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.836484 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.836402    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-envoy\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.881133 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.881065    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"]
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.937951    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-podinfo\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.938011    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-credential-socket\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.938038    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-data\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.938072    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/372dccb4-44bc-44c8-8d0b-819378f3f085-istiod-ca-cert\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.938140    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29jkz\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-kube-api-access-29jkz\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.938194    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-socket\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.938219    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-certs\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.938245    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-token\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.938684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.938313    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-envoy\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.939279 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.939131    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-certs\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.939357 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.939337    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-credential-socket\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.939431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.939411    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-data\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.939660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.939628    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-socket\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.940026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.940003    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/372dccb4-44bc-44c8-8d0b-819378f3f085-istiod-ca-cert\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.941582 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.941557    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-podinfo\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:32.941770 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:32.941751    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-envoy\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:33.035626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.035558    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29jkz\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-kube-api-access-29jkz\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:33.035626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.035591    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-token\") pod \"e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") " pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:33.055828 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.055788    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:33.496226 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.496191    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"]
Oct 30 19:20:33.497962 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:20:33.497935    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372dccb4_44bc_44c8_8d0b_819378f3f085.slice/crio-a9a6f831ef57ad88f8882e54a176e391ca69ea5c485979da3a2e692adffb5e9e WatchSource:0}: Error finding container a9a6f831ef57ad88f8882e54a176e391ca69ea5c485979da3a2e692adffb5e9e: Status 404 returned error can't find the container with id a9a6f831ef57ad88f8882e54a176e391ca69ea5c485979da3a2e692adffb5e9e
Oct 30 19:20:33.500422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.500389    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:33.500487 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.500457    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:33.500529 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.500485    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:33.855244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.855206    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" event={"ID":"372dccb4-44bc-44c8-8d0b-819378f3f085","Type":"ContainerStarted","Data":"985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4"}
Oct 30 19:20:33.855852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:33.855791    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" event={"ID":"372dccb4-44bc-44c8-8d0b-819378f3f085","Type":"ContainerStarted","Data":"a9a6f831ef57ad88f8882e54a176e391ca69ea5c485979da3a2e692adffb5e9e"}
Oct 30 19:20:34.057014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:34.056975    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:34.062968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:34.062942    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:34.181521 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:34.181410    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" podStartSLOduration=2.181396187 podStartE2EDuration="2.181396187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:33.986235911 +0000 UTC m=+2026.570718965" watchObservedRunningTime="2026-10-30 19:20:34.181396187 +0000 UTC m=+2026.765879226"
Oct 30 19:20:34.861951 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:34.861920    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:34.862862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:34.862806    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:20:46.189687 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:20:46.189653    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:20:55.322042 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.322000    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"]
Oct 30 19:20:55.322611 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.322597    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fcbf443-86fd-4ea5-9c23-f6d7c957be18" containerName="istio-proxy"
Oct 30 19:20:55.327432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.327410    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.381227 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.381188    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-7a790b-openshift-default-dockercfg-4rmb9\""
Oct 30 19:20:55.437064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437024    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fh2t\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-kube-api-access-5fh2t\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.437234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437141    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-podinfo\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.437234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437180    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c908ea0-f632-4c83-b6c9-440f60a50c99-istiod-ca-cert\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.437234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437202    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-socket\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.437354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437243    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-data\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.437354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437269    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-certs\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.437354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437290    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-credential-socket\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.437354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437316    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-token\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.437354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.437340    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-envoy\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.441945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.441904    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"]
Oct 30 19:20:55.538451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538421    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-data\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538467    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-certs\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538498    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-credential-socket\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538522    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-token\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538551    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-envoy\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538675    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fh2t\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-kube-api-access-5fh2t\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538832    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-podinfo\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538873    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-data\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538893    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c908ea0-f632-4c83-b6c9-440f60a50c99-istiod-ca-cert\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538917    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-certs\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538948    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-socket\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.538985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.538956    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-credential-socket\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.539363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.539221    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-socket\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.539465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.539449    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c908ea0-f632-4c83-b6c9-440f60a50c99-istiod-ca-cert\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.541399 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.541371    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-podinfo\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.541543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.541525    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-envoy\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.601968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.601932    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-token\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.602164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.602139    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fh2t\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-kube-api-access-5fh2t\") pod \"e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") " pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:55.685857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:55.685822    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:56.118751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:56.118724    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"]
Oct 30 19:20:56.121568 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:20:56.121541    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c908ea0_f632_4c83_b6c9_440f60a50c99.slice/crio-69087a6abdbd9b2b4bab242c4cf0d29be3070d35026760eee48183b89b894a97 WatchSource:0}: Error finding container 69087a6abdbd9b2b4bab242c4cf0d29be3070d35026760eee48183b89b894a97: Status 404 returned error can't find the container with id 69087a6abdbd9b2b4bab242c4cf0d29be3070d35026760eee48183b89b894a97
Oct 30 19:20:56.123828 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:56.123780    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:56.123921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:56.123904    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:56.123989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:56.123955    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:57.015670 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:57.015626    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" event={"ID":"1c908ea0-f632-4c83-b6c9-440f60a50c99","Type":"ContainerStarted","Data":"1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267"}
Oct 30 19:20:57.016270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:57.015727    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" event={"ID":"1c908ea0-f632-4c83-b6c9-440f60a50c99","Type":"ContainerStarted","Data":"69087a6abdbd9b2b4bab242c4cf0d29be3070d35026760eee48183b89b894a97"}
Oct 30 19:20:57.133022 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:57.132968    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" podStartSLOduration=2.132950931 podStartE2EDuration="2.132950931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:57.131872144 +0000 UTC m=+2049.716355141" watchObservedRunningTime="2026-10-30 19:20:57.132950931 +0000 UTC m=+2049.717433928"
Oct 30 19:20:57.686191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:57.686160    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:57.692122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:57.692097    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:58.022983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:58.022903    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:20:58.023898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:20:58.023877    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:21:16.202234 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:21:16.202191    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:21:23.936551 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:23.936510    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"]
Oct 30 19:21:23.940904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:23.940882    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:23.992096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:23.992059    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-539084-openshift-default-dockercfg-mfzgn\""
Oct 30 19:21:24.016798 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.016762    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jh4g\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-kube-api-access-9jh4g\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.017047 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.016935    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d60cfa3f-582b-4634-b75a-99344533b5b6-istiod-ca-cert\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.017047 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.016978    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-envoy\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.017047 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.017007    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-socket\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.017047 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.017045    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-certs\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.017309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.017140    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-data\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.017309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.017203    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-credential-socket\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.017309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.017263    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-podinfo\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.017309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.017299    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-token\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.042742 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.042708    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"]
Oct 30 19:21:24.119081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119048    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d60cfa3f-582b-4634-b75a-99344533b5b6-istiod-ca-cert\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.119347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119103    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-envoy\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.119347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119131    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-socket\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.119347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119154    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-certs\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.119347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119182    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-data\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.119347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119251    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-credential-socket\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.119347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119290    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-podinfo\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.119347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119329    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-token\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.120332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119420    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jh4g\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-kube-api-access-9jh4g\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.120332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119654    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-data\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.120332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119704    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-credential-socket\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.120332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119888    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-certs\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.120332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119951    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d60cfa3f-582b-4634-b75a-99344533b5b6-istiod-ca-cert\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.120332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.119988    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-socket\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.121847 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.121791    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-envoy\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.121984 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.121965    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-podinfo\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.226535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.215617    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-token\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.227112 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.218555    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jh4g\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-kube-api-access-9jh4g\") pod \"e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") " pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.259046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.257967    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:24.715458 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.715423    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"]
Oct 30 19:21:24.719726 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:21:24.719699    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60cfa3f_582b_4634_b75a_99344533b5b6.slice/crio-1534cfb98137f3ab3d1b9bba5cfd46c0d67eb03bebdd06144cb49dba18bcc4fa WatchSource:0}: Error finding container 1534cfb98137f3ab3d1b9bba5cfd46c0d67eb03bebdd06144cb49dba18bcc4fa: Status 404 returned error can't find the container with id 1534cfb98137f3ab3d1b9bba5cfd46c0d67eb03bebdd06144cb49dba18bcc4fa
Oct 30 19:21:24.722287 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.722251    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:24.722375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.722318    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:24.722375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:24.722349    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:25.250767 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:25.250727    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" event={"ID":"d60cfa3f-582b-4634-b75a-99344533b5b6","Type":"ContainerStarted","Data":"0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2"}
Oct 30 19:21:25.251181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:25.250796    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" event={"ID":"d60cfa3f-582b-4634-b75a-99344533b5b6","Type":"ContainerStarted","Data":"1534cfb98137f3ab3d1b9bba5cfd46c0d67eb03bebdd06144cb49dba18bcc4fa"}
Oct 30 19:21:25.259226 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:25.259200    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:25.265513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:25.265487    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:25.369783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:25.369718    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" podStartSLOduration=2.369703935 podStartE2EDuration="2.369703935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:25.367555551 +0000 UTC m=+2077.952038545" watchObservedRunningTime="2026-10-30 19:21:25.369703935 +0000 UTC m=+2077.954186938"
Oct 30 19:21:26.257524 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:26.257483    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:26.259168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:26.259139    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:21:35.068673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.068637    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"]
Oct 30 19:21:35.076672 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.076639    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.131040 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.131005    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-65d741-serving-cert\""
Oct 30 19:21:35.132496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.132465    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"]
Oct 30 19:21:35.169670 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.169631    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-metrics-certs\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.169872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.169785    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s2kq\" (UniqueName: \"kubernetes.io/projected/a85d8164-5351-4215-9aa7-2dc762f42675-kube-api-access-8s2kq\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.169929 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.169914    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85d8164-5351-4215-9aa7-2dc762f42675-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.169966 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.169941    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.271217 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.271171    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s2kq\" (UniqueName: \"kubernetes.io/projected/a85d8164-5351-4215-9aa7-2dc762f42675-kube-api-access-8s2kq\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.271433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.271310    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85d8164-5351-4215-9aa7-2dc762f42675-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.271433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.271347    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.271559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.271434    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-metrics-certs\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.271741 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:21:35.271714    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-serving-cert: secret "maas-api-e2e-ipp-b-65d741-serving-cert" not found
Oct 30 19:21:35.271881 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:21:35.271825    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls podName:a85d8164-5351-4215-9aa7-2dc762f42675 nodeName:}" failed. No retries permitted until 2026-10-30 19:21:35.771788904 +0000 UTC m=+2088.356271902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls") pod "maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr" (UID: "a85d8164-5351-4215-9aa7-2dc762f42675") : secret "maas-api-e2e-ipp-b-65d741-serving-cert" not found
Oct 30 19:21:35.272368 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.272342    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85d8164-5351-4215-9aa7-2dc762f42675-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.274195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.274175    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-metrics-certs\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.330758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.330690    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s2kq\" (UniqueName: \"kubernetes.io/projected/a85d8164-5351-4215-9aa7-2dc762f42675-kube-api-access-8s2kq\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.777780 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.777741    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.780289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.780254    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls\") pod \"maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:35.992638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:35.992602    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:36.374240 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:36.374201    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"]
Oct 30 19:21:36.378263 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:21:36.378238    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85d8164_5351_4215_9aa7_2dc762f42675.slice/crio-7e3ffc3f04250d964ac20aab146d808a4a8931568beb010950a498e154afe293 WatchSource:0}: Error finding container 7e3ffc3f04250d964ac20aab146d808a4a8931568beb010950a498e154afe293: Status 404 returned error can't find the container with id 7e3ffc3f04250d964ac20aab146d808a4a8931568beb010950a498e154afe293
Oct 30 19:21:37.333438 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:37.333400    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr" event={"ID":"a85d8164-5351-4215-9aa7-2dc762f42675","Type":"ContainerStarted","Data":"7e3ffc3f04250d964ac20aab146d808a4a8931568beb010950a498e154afe293"}
Oct 30 19:21:38.343509 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:38.343469    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr" event={"ID":"a85d8164-5351-4215-9aa7-2dc762f42675","Type":"ContainerStarted","Data":"d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528"}
Oct 30 19:21:38.344009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:38.343540    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:38.464266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:38.464207    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr" podStartSLOduration=2.510110712 podStartE2EDuration="3.464192884s" totalImagesPullingTime="954.082172ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:38.460575044 +0000 UTC m=+2091.045058043" watchObservedRunningTime="2026-10-30 19:21:38.464192884 +0000 UTC m=+2091.048675880"
Oct 30 19:21:44.359242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:44.359208    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:21:46.187948 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:21:46.187911    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:21:47.839974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:47.839940    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:21:47.844060 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:47.844034    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:21:47.852982 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:47.852955    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:21:47.856897 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:21:47.856870    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:22:00.539648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:00.539613    5232 generic.go:356] "Generic (PLEG): container finished" podID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerID="985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4" exitCode=0
Oct 30 19:22:00.540114 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:00.539685    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" event={"ID":"372dccb4-44bc-44c8-8d0b-819378f3f085","Type":"ContainerDied","Data":"985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4"}
Oct 30 19:22:00.540158 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:00.540115    5232 scope.go:122] "RemoveContainer" containerID="985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4"
Oct 30 19:22:00.541572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:00.541545    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:00.541630 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:00.541615    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:00.541668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:00.541645    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:01.568999 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:01.568957    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" event={"ID":"372dccb4-44bc-44c8-8d0b-819378f3f085","Type":"ContainerStarted","Data":"18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c"}
Oct 30 19:22:02.058216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:02.058176    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:22:02.062923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:02.062065    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:22:02.062923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:02.062147    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:22:03.061838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:03.061221    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:22:03.061838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:03.061298    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:22:04.062261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:04.062224    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:22:04.062778 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:04.062302    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:22:04.862487 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:04.862026    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:22:05.063225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:05.063185    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:22:05.063717 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:05.063243    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:22:06.058457 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:06.058417    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:22:06.058991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:06.058958    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:22:07.061601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:07.061573    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:22:07.062890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:07.062867    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:22:16.188600 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:22:16.188554    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:22:27.758163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:27.758128    5232 generic.go:356] "Generic (PLEG): container finished" podID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerID="1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267" exitCode=0
Oct 30 19:22:27.758591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:27.758194    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" event={"ID":"1c908ea0-f632-4c83-b6c9-440f60a50c99","Type":"ContainerDied","Data":"1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267"}
Oct 30 19:22:27.758633 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:27.758610    5232 scope.go:122] "RemoveContainer" containerID="1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267"
Oct 30 19:22:27.760242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:27.760213    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:27.760318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:27.760280    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:27.760318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:27.760312    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:28.023610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:28.023521    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:22:28.773953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:28.773910    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" event={"ID":"1c908ea0-f632-4c83-b6c9-440f60a50c99","Type":"ContainerStarted","Data":"e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd"}
Oct 30 19:22:29.686771 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:29.686735    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:22:29.688311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:29.688281    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:22:29.688426 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:29.688336    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:22:30.686732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:30.686696    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:22:30.687190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:30.686759    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:22:31.687157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:31.687125    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:22:31.687572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:31.687192    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:22:32.686400 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:32.686362    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:22:32.686582 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:32.686430    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:22:33.579432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.579275    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"]
Oct 30 19:22:33.583779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.583759    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.634450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.634419    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-539084-serving-cert\""
Oct 30 19:22:33.662837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.653624    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfhfj\" (UniqueName: \"kubernetes.io/projected/a4e80b7f-f318-4f87-9c51-856b871be5c6-kube-api-access-jfhfj\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.662837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.653867    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e80b7f-f318-4f87-9c51-856b871be5c6-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.662837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.653964    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.662837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.654084    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-metrics-certs\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.685684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.685651    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"]
Oct 30 19:22:33.687163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.687136    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:22:33.687310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.687196    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:22:33.755670 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.755630    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfhfj\" (UniqueName: \"kubernetes.io/projected/a4e80b7f-f318-4f87-9c51-856b871be5c6-kube-api-access-jfhfj\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.755920 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.755758    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e80b7f-f318-4f87-9c51-856b871be5c6-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.756000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.755932    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.756065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.756035    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-metrics-certs\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.756065 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:22:33.756050    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-serving-cert: secret "maas-api-e2e-ipp-b-539084-serving-cert" not found
Oct 30 19:22:33.756191 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:22:33.756113    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls podName:a4e80b7f-f318-4f87-9c51-856b871be5c6 nodeName:}" failed. No retries permitted until 2026-10-30 19:22:34.256095197 +0000 UTC m=+2146.840578183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls") pod "maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w" (UID: "a4e80b7f-f318-4f87-9c51-856b871be5c6") : secret "maas-api-e2e-ipp-b-539084-serving-cert" not found
Oct 30 19:22:33.756465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.756445    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e80b7f-f318-4f87-9c51-856b871be5c6-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.758650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.758618    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-metrics-certs\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:33.829649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:33.829554    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfhfj\" (UniqueName: \"kubernetes.io/projected/a4e80b7f-f318-4f87-9c51-856b871be5c6-kube-api-access-jfhfj\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:34.261987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:34.261942    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:34.264355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:34.264323    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls\") pod \"maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:34.497320 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:34.497287    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:34.692898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:34.692860    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:22:34.693378 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:34.693347    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:22:34.694079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:34.694059    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:22:34.883024 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:34.882970    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"]
Oct 30 19:22:34.894616 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:22:34.894583    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e80b7f_f318_4f87_9c51_856b871be5c6.slice/crio-19e8422a35446b52196502b5984be9aa0f71240832f0bcd3917572e96cb4c9c5 WatchSource:0}: Error finding container 19e8422a35446b52196502b5984be9aa0f71240832f0bcd3917572e96cb4c9c5: Status 404 returned error can't find the container with id 19e8422a35446b52196502b5984be9aa0f71240832f0bcd3917572e96cb4c9c5
Oct 30 19:22:35.838975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:35.838945    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w" event={"ID":"a4e80b7f-f318-4f87-9c51-856b871be5c6","Type":"ContainerStarted","Data":"19e8422a35446b52196502b5984be9aa0f71240832f0bcd3917572e96cb4c9c5"}
Oct 30 19:22:36.848892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:36.848847    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w" event={"ID":"a4e80b7f-f318-4f87-9c51-856b871be5c6","Type":"ContainerStarted","Data":"a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7"}
Oct 30 19:22:36.849310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:36.849091    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:36.969772 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:36.969666    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w" podStartSLOduration=3.052448323 podStartE2EDuration="3.969648077s" totalImagesPullingTime="917.199754ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:36.966007158 +0000 UTC m=+2149.550490166" watchObservedRunningTime="2026-10-30 19:22:36.969648077 +0000 UTC m=+2149.554131074"
Oct 30 19:22:42.861774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:42.861740    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:22:43.028612 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.028576    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"]
Oct 30 19:22:43.033550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.033524    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.083641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.083615    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-496c5b-serving-cert\""
Oct 30 19:22:43.099342 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.099292    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"]
Oct 30 19:22:43.174490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.174409    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c864f6-81b6-4d20-8f9a-600df2ad752b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.174637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.174490    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-metrics-certs\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.174637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.174550    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.174709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.174644    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx2rt\" (UniqueName: \"kubernetes.io/projected/81c864f6-81b6-4d20-8f9a-600df2ad752b-kube-api-access-dx2rt\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.276489 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.276434    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.276679 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.276557    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx2rt\" (UniqueName: \"kubernetes.io/projected/81c864f6-81b6-4d20-8f9a-600df2ad752b-kube-api-access-dx2rt\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.276679 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:22:43.276610    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-serving-cert: secret "maas-api-e2e-ipp-b-496c5b-serving-cert" not found
Oct 30 19:22:43.276873 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:22:43.276710    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls podName:81c864f6-81b6-4d20-8f9a-600df2ad752b nodeName:}" failed. No retries permitted until 2026-10-30 19:22:43.776678568 +0000 UTC m=+2156.361161565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls") pod "maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs" (UID: "81c864f6-81b6-4d20-8f9a-600df2ad752b") : secret "maas-api-e2e-ipp-b-496c5b-serving-cert" not found
Oct 30 19:22:43.276873 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.276732    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c864f6-81b6-4d20-8f9a-600df2ad752b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.276873 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.276855    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-metrics-certs\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.277953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.277925    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c864f6-81b6-4d20-8f9a-600df2ad752b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.280490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.280469    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-metrics-certs\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.335613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.335576    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx2rt\" (UniqueName: \"kubernetes.io/projected/81c864f6-81b6-4d20-8f9a-600df2ad752b-kube-api-access-dx2rt\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.783902 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.783862    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.786335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.786307    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:43.947970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:43.947931    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:44.327054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:44.326992    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"]
Oct 30 19:22:44.331641 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:22:44.331607    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c864f6_81b6_4d20_8f9a_600df2ad752b.slice/crio-5bc2f940d88d04798a7bb7403f2a2bbd212c1141d66de6e650a102d659628ac1 WatchSource:0}: Error finding container 5bc2f940d88d04798a7bb7403f2a2bbd212c1141d66de6e650a102d659628ac1: Status 404 returned error can't find the container with id 5bc2f940d88d04798a7bb7403f2a2bbd212c1141d66de6e650a102d659628ac1
Oct 30 19:22:44.911429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:44.911395    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs" event={"ID":"81c864f6-81b6-4d20-8f9a-600df2ad752b","Type":"ContainerStarted","Data":"5bc2f940d88d04798a7bb7403f2a2bbd212c1141d66de6e650a102d659628ac1"}
Oct 30 19:22:45.957002 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:45.956966    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs" event={"ID":"81c864f6-81b6-4d20-8f9a-600df2ad752b","Type":"ContainerStarted","Data":"0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb"}
Oct 30 19:22:45.957473 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:45.957046    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:46.073702 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:46.073647    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs" podStartSLOduration=3.268848489 podStartE2EDuration="4.073631251s" totalImagesPullingTime="804.782762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:46.070798004 +0000 UTC m=+2158.655281001" watchObservedRunningTime="2026-10-30 19:22:46.073631251 +0000 UTC m=+2158.658114248"
Oct 30 19:22:46.192854 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:22:46.192797    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:22:51.970059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:51.970026    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:22:56.259353 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:56.259321    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:22:56.259754 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:56.259375    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 19:22:57.042983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:57.042948    5232 generic.go:356] "Generic (PLEG): container finished" podID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerID="0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2" exitCode=0
Oct 30 19:22:57.043167 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:57.042990    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" event={"ID":"d60cfa3f-582b-4634-b75a-99344533b5b6","Type":"ContainerDied","Data":"0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2"}
Oct 30 19:22:57.043432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:57.043418    5232 scope.go:122] "RemoveContainer" containerID="0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2"
Oct 30 19:22:57.045090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:57.045057    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:57.045152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:57.045131    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:57.045188 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:57.045165    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:22:58.068806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:58.068769    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" event={"ID":"d60cfa3f-582b-4634-b75a-99344533b5b6","Type":"ContainerStarted","Data":"102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd"}
Oct 30 19:22:58.280388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:58.270033    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:22:58.280388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:58.279885    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:22:58.280388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:58.280100    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 19:22:59.261948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:59.260184    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:22:59.261948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:22:59.260245    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 19:23:00.262272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:00.262225    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:23:00.262695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:00.262304    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 19:23:01.262246 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:01.262202    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:23:01.262452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:01.262267    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 19:23:02.259313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:02.259258    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:23:02.259544 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:02.259352    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 30 19:23:03.263755 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:03.263725    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:23:03.264186 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:03.264164    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:23:03.264843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:03.264826    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:23:16.188502 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:23:16.188464    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:23:46.188082 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:23:46.188047    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:23:48.512035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:48.512000    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"]
Oct 30 19:23:48.512439 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:48.512245    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy" containerID="cri-o://3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8" gracePeriod=30
Oct 30 19:23:53.791578 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.791512    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:23:53.978272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.978228    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-kube-api-access-z6hqd\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-kube-api-access-z6hqd\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.981836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.978577    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-socket\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.982092 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.982068    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-envoy\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.982262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.982247    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-podinfo\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.988989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.988951    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-token\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-token\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.989126 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.989008    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-credential-socket\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.989126 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.989039    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-data\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.989126 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.989113    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18349e05-80cb-4f86-abf7-84bb969e7f50-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18349e05-80cb-4f86-abf7-84bb969e7f50-istiod-ca-cert\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.989297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.989229    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-certs\") pod \"18349e05-80cb-4f86-abf7-84bb969e7f50\" (UID: \"18349e05-80cb-4f86-abf7-84bb969e7f50\") "
Oct 30 19:23:53.989763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.988425    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-socket" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:53.989763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.989216    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-kube-api-access-z6hqd" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "kube-api-access-z6hqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:53.989763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.989432    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-credential-socket" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:53.989763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.989740    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-data" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:53.990096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.989869    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18349e05-80cb-4f86-abf7-84bb969e7f50-istiod-ca-cert" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:53.990096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.990058    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-certs" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:53.991985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.991964    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-podinfo" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:23:53.992145 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.992125    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-token" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:53.993138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.993117    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-envoy" pod "18349e05-80cb-4f86-abf7-84bb969e7f50" (UID: "18349e05-80cb-4f86-abf7-84bb969e7f50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:53.996143 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996120    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:53.996264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996247    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:53.996350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996268    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:53.996350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996282    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:53.996350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996296    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:53.996350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996309    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18349e05-80cb-4f86-abf7-84bb969e7f50-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:53.996350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996323    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:53.996350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996337    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6hqd\" (UniqueName: \"kubernetes.io/projected/18349e05-80cb-4f86-abf7-84bb969e7f50-kube-api-access-z6hqd\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:53.996557 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:53.996352    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18349e05-80cb-4f86-abf7-84bb969e7f50-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:54.472797 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.472765    5232 generic.go:356] "Generic (PLEG): container finished" podID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerID="3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8" exitCode=0
Oct 30 19:23:54.472978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.472843    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"
Oct 30 19:23:54.472978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.472843    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" event={"ID":"18349e05-80cb-4f86-abf7-84bb969e7f50","Type":"ContainerDied","Data":"3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8"}
Oct 30 19:23:54.472978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.472905    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m" event={"ID":"18349e05-80cb-4f86-abf7-84bb969e7f50","Type":"ContainerDied","Data":"a71324adad682e391efe0d7cfcb9ac07a8ef7f9926978c41216db3ede945e0ca"}
Oct 30 19:23:54.472978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.472934    5232 scope.go:122] "RemoveContainer" containerID="3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8"
Oct 30 19:23:54.484620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.484593    5232 scope.go:122] "RemoveContainer" containerID="ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040"
Oct 30 19:23:54.493781 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.493761    5232 scope.go:122] "RemoveContainer" containerID="3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8"
Oct 30 19:23:54.494109 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:23:54.494088    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8\": container with ID starting with 3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8 not found: ID does not exist" containerID="3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8"
Oct 30 19:23:54.494203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.494120    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8"} err="failed to get container status \"3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8\": rpc error: code = NotFound desc = could not find container \"3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8\": container with ID starting with 3730414c8a623a9bafb7a4a4d1edcd8635acd2aa8a8492a2b490136caa89b1c8 not found: ID does not exist"
Oct 30 19:23:54.494203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.494146    5232 scope.go:122] "RemoveContainer" containerID="ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040"
Oct 30 19:23:54.494430 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:23:54.494405    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040\": container with ID starting with ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040 not found: ID does not exist" containerID="ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040"
Oct 30 19:23:54.494513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.494433    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040"} err="failed to get container status \"ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040\": rpc error: code = NotFound desc = could not find container \"ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040\": container with ID starting with ec120c95484f887859b7519da3749b93ff6599559b2eec48f47f2ea9968df040 not found: ID does not exist"
Oct 30 19:23:54.680422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.680388    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"]
Oct 30 19:23:54.728075 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:54.727993    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-fb4a1b-openshift-default-7fb74948dc-gkv6m"]
Oct 30 19:23:55.540939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:23:55.540906    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" path="/var/lib/kubelet/pods/18349e05-80cb-4f86-abf7-84bb969e7f50/volumes"
Oct 30 19:24:16.188670 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:24:16.188621    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:24:20.242177 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:20.242141    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"]
Oct 30 19:24:20.242699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:20.242343    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" containerID="cri-o://69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac" gracePeriod=30
Oct 30 19:24:23.036454 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:23.036418    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"]
Oct 30 19:24:23.036893 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:23.036625    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" containerID="cri-o://144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764" gracePeriod=30
Oct 30 19:24:23.665424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:23.665385    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:24:23.665618 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:23.665449    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:24:24.469067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:24.469028    5232 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:24:24.469449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:24.469082    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:24:25.516191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.516129    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:24:25.639417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639390    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istiod-ca-cert\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.639590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639428    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-podinfo\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.639590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639560    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-token\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-token\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.639668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639610    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-socket\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.639668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639662    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-envoy\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.639776 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639707    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-data\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.639776 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639735    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-credential-socket\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.639919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639901    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-kube-api-access-9k2sk\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-kube-api-access-9k2sk\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.639978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639945    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-certs\") pod \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\" (UID: \"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229\") "
Oct 30 19:24:25.640043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639946    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-socket" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:25.640043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.639995    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istiod-ca-cert" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:25.640257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.640175    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-credential-socket" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:25.640392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.640313    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-data" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:25.640441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.640386    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-certs" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:25.640717 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.640693    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.640834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.640722    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.640834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.640739    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.640834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.640755    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.640834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.640770    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.641841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.641804    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-podinfo" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:24:25.642190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.642169    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-envoy" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:25.642254 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.642195    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-token" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:25.642254 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.642219    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-kube-api-access-9k2sk" pod "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" (UID: "7dbcebf7-74ce-4710-9dc3-a1bc64ba4229"). InnerVolumeSpecName "kube-api-access-9k2sk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:25.677718 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.677682    5232 generic.go:356] "Generic (PLEG): container finished" podID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerID="69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac" exitCode=0
Oct 30 19:24:25.677910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.677733    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"
Oct 30 19:24:25.677910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.677757    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" event={"ID":"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229","Type":"ContainerDied","Data":"69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac"}
Oct 30 19:24:25.677910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.677807    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh" event={"ID":"7dbcebf7-74ce-4710-9dc3-a1bc64ba4229","Type":"ContainerDied","Data":"83d0515866a6a416d38e54774ca1fd32fa73982049a51b1e1f68eb2a6caa4769"}
Oct 30 19:24:25.677910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.677840    5232 scope.go:122] "RemoveContainer" containerID="69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac"
Oct 30 19:24:25.688397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.688377    5232 scope.go:122] "RemoveContainer" containerID="69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac"
Oct 30 19:24:25.688643 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:24:25.688625    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac\": container with ID starting with 69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac not found: ID does not exist" containerID="69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac"
Oct 30 19:24:25.688696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.688651    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac"} err="failed to get container status \"69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac\": rpc error: code = NotFound desc = could not find container \"69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac\": container with ID starting with 69460fd4230158bb0dc72fcf662fdb99855f6c39e48b31df4806f83f5c020fac not found: ID does not exist"
Oct 30 19:24:25.741784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.741753    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k2sk\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-kube-api-access-9k2sk\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.741784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.741782    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.742073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.741799    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.742073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.741808    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:25.818599 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.818510    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"]
Oct 30 19:24:25.867264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:25.867230    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-q44kh"]
Oct 30 19:24:27.549752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:27.549677    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" path="/var/lib/kubelet/pods/7dbcebf7-74ce-4710-9dc3-a1bc64ba4229/volumes"
Oct 30 19:24:28.291632 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.291606    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:24:28.375601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.375565    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-kube-api-access-f6h94\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-kube-api-access-f6h94\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.375778 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.375645    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-podinfo\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.375778 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.375684    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istiod-ca-cert\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.375905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.375783    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-certs\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.375905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.375870    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-socket\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.376009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.375920    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-token\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-token\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.376009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.375955    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-credential-socket\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.376009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.375999    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-envoy\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.376171 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376048    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-data\") pod \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\" (UID: \"c1312ab0-a901-44e5-84ff-0a59ec6b59a5\") "
Oct 30 19:24:28.376171 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376149    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-certs" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:28.376283 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376195    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-socket" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:28.376283 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376226    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istiod-ca-cert" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:28.376395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376375    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-credential-socket" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:28.376548 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376524    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-data" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:28.376752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376733    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.376831 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376760    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.376831 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376776    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.376831 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376790    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.376831 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.376804    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.377914 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.377895    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-kube-api-access-f6h94" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "kube-api-access-f6h94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:28.378181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.378157    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-token" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:28.378289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.378262    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-podinfo" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:24:28.378380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.378357    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-envoy" pod "c1312ab0-a901-44e5-84ff-0a59ec6b59a5" (UID: "c1312ab0-a901-44e5-84ff-0a59ec6b59a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:28.477947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.477915    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.477947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.477943    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.477947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.477953    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6h94\" (UniqueName: \"kubernetes.io/projected/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-kube-api-access-f6h94\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.478180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.477962    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1312ab0-a901-44e5-84ff-0a59ec6b59a5-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:28.701638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.701551    5232 generic.go:356] "Generic (PLEG): container finished" podID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerID="144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764" exitCode=0
Oct 30 19:24:28.701638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.701599    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"
Oct 30 19:24:28.702174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.701642    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" event={"ID":"c1312ab0-a901-44e5-84ff-0a59ec6b59a5","Type":"ContainerDied","Data":"144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764"}
Oct 30 19:24:28.702174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.701702    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn" event={"ID":"c1312ab0-a901-44e5-84ff-0a59ec6b59a5","Type":"ContainerDied","Data":"d1b663cbfb22d4521b65db717cd084483f2c118faaeb34f7eec22d0e4c8bd87b"}
Oct 30 19:24:28.702174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.701732    5232 scope.go:122] "RemoveContainer" containerID="144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764"
Oct 30 19:24:28.714234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.714222    5232 scope.go:122] "RemoveContainer" containerID="144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764"
Oct 30 19:24:28.714471 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:24:28.714445    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764\": container with ID starting with 144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764 not found: ID does not exist" containerID="144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764"
Oct 30 19:24:28.714576 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.714480    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764"} err="failed to get container status \"144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764\": rpc error: code = NotFound desc = could not find container \"144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764\": container with ID starting with 144e4dbf3295fd3f7dd2f37397f82412511cea195cb4c8131b0f9e242eef9764 not found: ID does not exist"
Oct 30 19:24:28.864821 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.864771    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"]
Oct 30 19:24:28.912554 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:28.912520    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-9j7nn"]
Oct 30 19:24:29.539926 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:24:29.539893    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" path="/var/lib/kubelet/pods/c1312ab0-a901-44e5-84ff-0a59ec6b59a5/volumes"
Oct 30 19:24:46.188035 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:24:46.187996    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:25:16.188543 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:25:16.188505    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:25:16.995531 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:16.995497    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"]
Oct 30 19:25:16.995765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:16.995736    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr" podUID="a85d8164-5351-4215-9aa7-2dc762f42675" containerName="maas-api" containerID="cri-o://d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528" gracePeriod=30
Oct 30 19:25:17.283050 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.283022    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:25:17.406205 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.406169    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a85d8164-5351-4215-9aa7-2dc762f42675-kube-api-access-8s2kq\" (UniqueName: \"kubernetes.io/projected/a85d8164-5351-4215-9aa7-2dc762f42675-kube-api-access-8s2kq\") pod \"a85d8164-5351-4215-9aa7-2dc762f42675\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") "
Oct 30 19:25:17.406393 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.406235    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls\") pod \"a85d8164-5351-4215-9aa7-2dc762f42675\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") "
Oct 30 19:25:17.406393 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.406316    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a85d8164-5351-4215-9aa7-2dc762f42675-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85d8164-5351-4215-9aa7-2dc762f42675-openshift-service-ca\") pod \"a85d8164-5351-4215-9aa7-2dc762f42675\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") "
Oct 30 19:25:17.406525 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.406500    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-metrics-certs\") pod \"a85d8164-5351-4215-9aa7-2dc762f42675\" (UID: \"a85d8164-5351-4215-9aa7-2dc762f42675\") "
Oct 30 19:25:17.408001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.407956    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a85d8164-5351-4215-9aa7-2dc762f42675-openshift-service-ca" pod "a85d8164-5351-4215-9aa7-2dc762f42675" (UID: "a85d8164-5351-4215-9aa7-2dc762f42675"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:17.409571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.409544    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls" pod "a85d8164-5351-4215-9aa7-2dc762f42675" (UID: "a85d8164-5351-4215-9aa7-2dc762f42675"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:17.409677 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.409660    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-metrics-certs" pod "a85d8164-5351-4215-9aa7-2dc762f42675" (UID: "a85d8164-5351-4215-9aa7-2dc762f42675"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:17.410860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.410834    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a85d8164-5351-4215-9aa7-2dc762f42675-kube-api-access-8s2kq" pod "a85d8164-5351-4215-9aa7-2dc762f42675" (UID: "a85d8164-5351-4215-9aa7-2dc762f42675"). InnerVolumeSpecName "kube-api-access-8s2kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:17.508577 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.508477    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s2kq\" (UniqueName: \"kubernetes.io/projected/a85d8164-5351-4215-9aa7-2dc762f42675-kube-api-access-8s2kq\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:17.508577 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.508505    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:17.508577 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.508517    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85d8164-5351-4215-9aa7-2dc762f42675-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:17.508577 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:17.508525    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a85d8164-5351-4215-9aa7-2dc762f42675-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:18.094703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.094663    5232 generic.go:356] "Generic (PLEG): container finished" podID="a85d8164-5351-4215-9aa7-2dc762f42675" containerID="d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528" exitCode=0
Oct 30 19:25:18.094950 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.094752    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr" event={"ID":"a85d8164-5351-4215-9aa7-2dc762f42675","Type":"ContainerDied","Data":"d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528"}
Oct 30 19:25:18.094950 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.094806    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr" event={"ID":"a85d8164-5351-4215-9aa7-2dc762f42675","Type":"ContainerDied","Data":"7e3ffc3f04250d964ac20aab146d808a4a8931568beb010950a498e154afe293"}
Oct 30 19:25:18.094950 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.094849    5232 scope.go:122] "RemoveContainer" containerID="d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528"
Oct 30 19:25:18.095135 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.095024    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"
Oct 30 19:25:18.109313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.109286    5232 scope.go:122] "RemoveContainer" containerID="d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528"
Oct 30 19:25:18.109676 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:25:18.109651    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528\": container with ID starting with d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528 not found: ID does not exist" containerID="d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528"
Oct 30 19:25:18.109770 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.109689    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528"} err="failed to get container status \"d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528\": rpc error: code = NotFound desc = could not find container \"d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528\": container with ID starting with d79cb7d459f79011231c2f97d69671325f2fe404ad95a92399922871b13a6528 not found: ID does not exist"
Oct 30 19:25:18.255552 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.255516    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"]
Oct 30 19:25:18.302478 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:18.302436    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-65d741-65d879db87-p9fgr"]
Oct 30 19:25:19.546475 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:19.546439    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a85d8164-5351-4215-9aa7-2dc762f42675" path="/var/lib/kubelet/pods/a85d8164-5351-4215-9aa7-2dc762f42675/volumes"
Oct 30 19:25:26.210888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:26.210853    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"]
Oct 30 19:25:26.211344 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:26.211109    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy" containerID="cri-o://18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c" gracePeriod=30
Oct 30 19:25:28.339720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.339672    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"]
Oct 30 19:25:28.340849 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.340806    5232 cpu_manager.go:414] "containerMap: removing container" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy"
Oct 30 19:25:28.340967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.340851    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy"
Oct 30 19:25:28.340967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.340907    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy"
Oct 30 19:25:28.340967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.340915    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy"
Oct 30 19:25:28.340967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.340939    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a85d8164-5351-4215-9aa7-2dc762f42675" containerName="maas-api"
Oct 30 19:25:28.340967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.340947    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a85d8164-5351-4215-9aa7-2dc762f42675" containerName="maas-api"
Oct 30 19:25:28.340967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.340961    5232 cpu_manager.go:414] "containerMap: removing container" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy"
Oct 30 19:25:28.340967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.340968    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy"
Oct 30 19:25:28.341290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.341025    5232 cpu_manager.go:414] "containerMap: removing container" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy"
Oct 30 19:25:28.341290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.341032    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy"
Oct 30 19:25:28.341290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.341264    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1312ab0-a901-44e5-84ff-0a59ec6b59a5" containerName="istio-proxy"
Oct 30 19:25:28.341290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.341277    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a85d8164-5351-4215-9aa7-2dc762f42675" containerName="maas-api"
Oct 30 19:25:28.341479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.341293    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy"
Oct 30 19:25:28.341479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.341307    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="18349e05-80cb-4f86-abf7-84bb969e7f50" containerName="istio-proxy"
Oct 30 19:25:28.341479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.341332    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="7dbcebf7-74ce-4710-9dc3-a1bc64ba4229" containerName="istio-proxy"
Oct 30 19:25:28.347705 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.347682    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.400170 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.400123    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-4833f3-openshift-default-dockercfg-nt6qv\""
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450254    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-credential-socket\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450398    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-envoy\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450441    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-istio-token\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450534    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50030bcc-3a51-420a-8542-fde4481596db-istio-podinfo\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450566    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4wwj\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-kube-api-access-v4wwj\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450596    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-certs\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450697    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-socket\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450722    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-data\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.450839    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50030bcc-3a51-420a-8542-fde4481596db-istiod-ca-cert\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.459867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.458162    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"]
Oct 30 19:25:28.551763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.551727    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50030bcc-3a51-420a-8542-fde4481596db-istio-podinfo\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.551763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.551769    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4wwj\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-kube-api-access-v4wwj\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.551795    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-certs\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.551893    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-socket\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.551919    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-data\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.551999    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50030bcc-3a51-420a-8542-fde4481596db-istiod-ca-cert\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.552057    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-credential-socket\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.552130    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-envoy\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.552170    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-istio-token\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.552290    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-certs\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552512 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.552391    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-socket\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552512 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.552406    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-data\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552608 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.552522    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-credential-socket\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.552773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.552754    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50030bcc-3a51-420a-8542-fde4481596db-istiod-ca-cert\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.554395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.554373    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-envoy\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.554518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.554407    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50030bcc-3a51-420a-8542-fde4481596db-istio-podinfo\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.619906 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.619806    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-istio-token\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.620056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.619969    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4wwj\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-kube-api-access-v4wwj\") pod \"e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") " pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:28.671335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:28.671297    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:29.105148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:29.105111    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"]
Oct 30 19:25:29.109314 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:25:29.109273    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50030bcc_3a51_420a_8542_fde4481596db.slice/crio-7ae654114eaa4d8540710282bea0fc219f2afc394731ebda63d5163eb49ce99a WatchSource:0}: Error finding container 7ae654114eaa4d8540710282bea0fc219f2afc394731ebda63d5163eb49ce99a: Status 404 returned error can't find the container with id 7ae654114eaa4d8540710282bea0fc219f2afc394731ebda63d5163eb49ce99a
Oct 30 19:25:29.111888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:29.111849    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:25:29.111967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:29.111930    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:25:29.111967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:29.111960    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:25:29.170433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:29.170405    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" event={"ID":"50030bcc-3a51-420a-8542-fde4481596db","Type":"ContainerStarted","Data":"7ae654114eaa4d8540710282bea0fc219f2afc394731ebda63d5163eb49ce99a"}
Oct 30 19:25:30.178266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:30.178231    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" event={"ID":"50030bcc-3a51-420a-8542-fde4481596db","Type":"ContainerStarted","Data":"111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396"}
Oct 30 19:25:30.344325 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:30.344274    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podStartSLOduration=2.344257909 podStartE2EDuration="2.344257909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:30.34305124 +0000 UTC m=+2322.927534237" watchObservedRunningTime="2026-10-30 19:25:30.344257909 +0000 UTC m=+2322.928740906"
Oct 30 19:25:30.672469 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:30.672435    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:30.678676 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:30.678648    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:31.184692 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.184664    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:31.185516 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.185499    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:25:31.488027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.488000    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.587831    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-token\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-token\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.587955    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-credential-socket\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.588018    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-socket\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.588090    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-kube-api-access-29jkz\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-kube-api-access-29jkz\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.588127    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/372dccb4-44bc-44c8-8d0b-819378f3f085-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/372dccb4-44bc-44c8-8d0b-819378f3f085-istiod-ca-cert\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.588192    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-podinfo\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.588251    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-certs\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.588298    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-envoy\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.588361    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-data\") pod \"372dccb4-44bc-44c8-8d0b-819378f3f085\" (UID: \"372dccb4-44bc-44c8-8d0b-819378f3f085\") "
Oct 30 19:25:31.591105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.589383    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-data" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:25:31.591776 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.591633    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-credential-socket" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:25:31.592748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.591843    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372dccb4-44bc-44c8-8d0b-819378f3f085-istiod-ca-cert" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:31.592748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.591856    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-socket" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:25:31.592748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.592323    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-certs" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:25:31.592748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.592653    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-token" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:31.597720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.597694    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-podinfo" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:25:31.597862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.597785    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-kube-api-access-29jkz" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "kube-api-access-29jkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:31.599056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.599033    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-envoy" pod "372dccb4-44bc-44c8-8d0b-819378f3f085" (UID: "372dccb4-44bc-44c8-8d0b-819378f3f085"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:25:31.690196 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690153    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.690196 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690189    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.690196 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690200    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.690441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690210    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.690441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690221    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.690441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690230    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29jkz\" (UniqueName: \"kubernetes.io/projected/372dccb4-44bc-44c8-8d0b-819378f3f085-kube-api-access-29jkz\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.690441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690240    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/372dccb4-44bc-44c8-8d0b-819378f3f085-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.690441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690249    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/372dccb4-44bc-44c8-8d0b-819378f3f085-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.690441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:31.690257    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/372dccb4-44bc-44c8-8d0b-819378f3f085-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:32.193138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.193097    5232 generic.go:356] "Generic (PLEG): container finished" podID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerID="18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c" exitCode=0
Oct 30 19:25:32.193559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.193164    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"
Oct 30 19:25:32.193559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.193186    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" event={"ID":"372dccb4-44bc-44c8-8d0b-819378f3f085","Type":"ContainerDied","Data":"18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c"}
Oct 30 19:25:32.193559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.193269    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9" event={"ID":"372dccb4-44bc-44c8-8d0b-819378f3f085","Type":"ContainerDied","Data":"a9a6f831ef57ad88f8882e54a176e391ca69ea5c485979da3a2e692adffb5e9e"}
Oct 30 19:25:32.193559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.193298    5232 scope.go:122] "RemoveContainer" containerID="18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c"
Oct 30 19:25:32.203452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.203429    5232 scope.go:122] "RemoveContainer" containerID="985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4"
Oct 30 19:25:32.215942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.215924    5232 scope.go:122] "RemoveContainer" containerID="18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c"
Oct 30 19:25:32.216234 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:25:32.216211    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c\": container with ID starting with 18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c not found: ID does not exist" containerID="18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c"
Oct 30 19:25:32.216292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.216245    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c"} err="failed to get container status \"18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c\": rpc error: code = NotFound desc = could not find container \"18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c\": container with ID starting with 18cac35d6af686afb4a9d9becf2b3cbd805779bfe6d19d5f4216bc981666158c not found: ID does not exist"
Oct 30 19:25:32.216292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.216264    5232 scope.go:122] "RemoveContainer" containerID="985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4"
Oct 30 19:25:32.216509 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:25:32.216489    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4\": container with ID starting with 985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4 not found: ID does not exist" containerID="985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4"
Oct 30 19:25:32.216559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.216515    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4"} err="failed to get container status \"985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4\": rpc error: code = NotFound desc = could not find container \"985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4\": container with ID starting with 985dc79feae79c76509958eca4a6271781188fb4314f622b748e67d69c2352f4 not found: ID does not exist"
Oct 30 19:25:32.353314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.353206    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"]
Oct 30 19:25:32.357054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:32.357031    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-65d741-openshift-default-7d9956f967-d2zw9"]
Oct 30 19:25:33.540070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:25:33.540038    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" path="/var/lib/kubelet/pods/372dccb4-44bc-44c8-8d0b-819378f3f085/volumes"
Oct 30 19:25:46.192851 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:25:46.192792    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:26:16.188258 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:26:16.188218    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:26:46.187480 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:26:46.187445    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:26:47.888253 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:26:47.888226    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:26:47.892654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:26:47.892630    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:26:47.903236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:26:47.903214    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:26:47.907622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:26:47.907605    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:27:00.119243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.119203    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"]
Oct 30 19:27:00.119744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.119447    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb" podUID="90484b62-de48-4b7a-8e3b-da5c11f9862d" containerName="discovery" containerID="cri-o://0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9" gracePeriod=30
Oct 30 19:27:00.192899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.192866    5232 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-nqjqb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.104:8080/ready\": dial tcp 10.133.0.104:8080: connect: connection refused" start-of-body=
Oct 30 19:27:00.193028 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.192944    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb" podUID="90484b62-de48-4b7a-8e3b-da5c11f9862d" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.104:8080/ready\": dial tcp 10.133.0.104:8080: connect: connection refused"
Oct 30 19:27:00.388032 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.387969    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:27:00.477833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.468349    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-cacerts\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-cacerts\") pod \"90484b62-de48-4b7a-8e3b-da5c11f9862d\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") "
Oct 30 19:27:00.477833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.468424    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-kubeconfig\") pod \"90484b62-de48-4b7a-8e3b-da5c11f9862d\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") "
Oct 30 19:27:00.477833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.468455    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-dns-cert\") pod \"90484b62-de48-4b7a-8e3b-da5c11f9862d\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") "
Oct 30 19:27:00.477833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.468502    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90484b62-de48-4b7a-8e3b-da5c11f9862d-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/90484b62-de48-4b7a-8e3b-da5c11f9862d-local-certs\") pod \"90484b62-de48-4b7a-8e3b-da5c11f9862d\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") "
Oct 30 19:27:00.477833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.468613    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-kube-api-access-nmgbc\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-kube-api-access-nmgbc\") pod \"90484b62-de48-4b7a-8e3b-da5c11f9862d\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") "
Oct 30 19:27:00.477833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.468665    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-ca-configmap\") pod \"90484b62-de48-4b7a-8e3b-da5c11f9862d\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") "
Oct 30 19:27:00.477833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.468704    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-token\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-token\") pod \"90484b62-de48-4b7a-8e3b-da5c11f9862d\" (UID: \"90484b62-de48-4b7a-8e3b-da5c11f9862d\") "
Oct 30 19:27:00.477833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.471785    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-ca-configmap" pod "90484b62-de48-4b7a-8e3b-da5c11f9862d" (UID: "90484b62-de48-4b7a-8e3b-da5c11f9862d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:00.496297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.495257    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-cacerts" pod "90484b62-de48-4b7a-8e3b-da5c11f9862d" (UID: "90484b62-de48-4b7a-8e3b-da5c11f9862d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:00.496297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.495266    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-kube-api-access-nmgbc" pod "90484b62-de48-4b7a-8e3b-da5c11f9862d" (UID: "90484b62-de48-4b7a-8e3b-da5c11f9862d"). InnerVolumeSpecName "kube-api-access-nmgbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:00.496297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.495335    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-token" pod "90484b62-de48-4b7a-8e3b-da5c11f9862d" (UID: "90484b62-de48-4b7a-8e3b-da5c11f9862d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:00.496297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.495402    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-dns-cert" pod "90484b62-de48-4b7a-8e3b-da5c11f9862d" (UID: "90484b62-de48-4b7a-8e3b-da5c11f9862d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:00.498246 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.498204    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-kubeconfig" pod "90484b62-de48-4b7a-8e3b-da5c11f9862d" (UID: "90484b62-de48-4b7a-8e3b-da5c11f9862d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:00.498403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.498306    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90484b62-de48-4b7a-8e3b-da5c11f9862d-local-certs" pod "90484b62-de48-4b7a-8e3b-da5c11f9862d" (UID: "90484b62-de48-4b7a-8e3b-da5c11f9862d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:00.570451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.570416    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmgbc\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-kube-api-access-nmgbc\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:00.570451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.570447    5232 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-ca-configmap\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:00.570451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.570458    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:00.570685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.570467    5232 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-cacerts\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:00.570685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.570475    5232 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-kubeconfig\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:00.570685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.570484    5232 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/90484b62-de48-4b7a-8e3b-da5c11f9862d-istio-csr-dns-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:00.570685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.570492    5232 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/90484b62-de48-4b7a-8e3b-da5c11f9862d-local-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:00.786916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.786804    5232 generic.go:356] "Generic (PLEG): container finished" podID="90484b62-de48-4b7a-8e3b-da5c11f9862d" containerID="0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9" exitCode=0
Oct 30 19:27:00.786916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.786870    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"
Oct 30 19:27:00.786916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.786887    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb" event={"ID":"90484b62-de48-4b7a-8e3b-da5c11f9862d","Type":"ContainerDied","Data":"0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9"}
Oct 30 19:27:00.787187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.786943    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb" event={"ID":"90484b62-de48-4b7a-8e3b-da5c11f9862d","Type":"ContainerDied","Data":"be14e4ee2237e72afac1a93d56096ca0d1ff9eead011e16a07f0a2df58a18eac"}
Oct 30 19:27:00.787187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.786969    5232 scope.go:122] "RemoveContainer" containerID="0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9"
Oct 30 19:27:00.797503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.797484    5232 scope.go:122] "RemoveContainer" containerID="0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9"
Oct 30 19:27:00.797749 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:27:00.797728    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9\": container with ID starting with 0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9 not found: ID does not exist" containerID="0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9"
Oct 30 19:27:00.797797 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.797757    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9"} err="failed to get container status \"0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9\": rpc error: code = NotFound desc = could not find container \"0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9\": container with ID starting with 0e3b9edafd5d37b8e0af62bb330d40fba18e89b62b28c407cd86fdc00e7d59a9 not found: ID does not exist"
Oct 30 19:27:00.953225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:00.953147    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"]
Oct 30 19:27:01.002047 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:01.002013    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nqjqb"]
Oct 30 19:27:01.540858 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:01.540752    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90484b62-de48-4b7a-8e3b-da5c11f9862d" path="/var/lib/kubelet/pods/90484b62-de48-4b7a-8e3b-da5c11f9862d/volumes"
Oct 30 19:27:07.053324 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.053284    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"]
Oct 30 19:27:07.053890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.053537    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl" podUID="8327232a-4d38-4266-a5e8-dc464ea7fb0a" containerName="maas-api" containerID="cri-o://63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf" gracePeriod=30
Oct 30 19:27:07.325266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.325238    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:27:07.411769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.411714    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"]
Oct 30 19:27:07.412187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.412148    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy" containerID="cri-o://d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505" gracePeriod=30
Oct 30 19:27:07.466926 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.466887    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8327232a-4d38-4266-a5e8-dc464ea7fb0a-kube-api-access-pvl8n\" (UniqueName: \"kubernetes.io/projected/8327232a-4d38-4266-a5e8-dc464ea7fb0a-kube-api-access-pvl8n\") pod \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") "
Oct 30 19:27:07.467137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.466976    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-metrics-certs\") pod \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") "
Oct 30 19:27:07.467137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.467078    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8327232a-4d38-4266-a5e8-dc464ea7fb0a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8327232a-4d38-4266-a5e8-dc464ea7fb0a-openshift-service-ca\") pod \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") "
Oct 30 19:27:07.467286 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.467233    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls\") pod \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\" (UID: \"8327232a-4d38-4266-a5e8-dc464ea7fb0a\") "
Oct 30 19:27:07.469020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.468986    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8327232a-4d38-4266-a5e8-dc464ea7fb0a-openshift-service-ca" pod "8327232a-4d38-4266-a5e8-dc464ea7fb0a" (UID: "8327232a-4d38-4266-a5e8-dc464ea7fb0a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:07.471105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.471079    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls" pod "8327232a-4d38-4266-a5e8-dc464ea7fb0a" (UID: "8327232a-4d38-4266-a5e8-dc464ea7fb0a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:07.471197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.471112    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-metrics-certs" pod "8327232a-4d38-4266-a5e8-dc464ea7fb0a" (UID: "8327232a-4d38-4266-a5e8-dc464ea7fb0a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:07.472963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.472937    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8327232a-4d38-4266-a5e8-dc464ea7fb0a-kube-api-access-pvl8n" pod "8327232a-4d38-4266-a5e8-dc464ea7fb0a" (UID: "8327232a-4d38-4266-a5e8-dc464ea7fb0a"). InnerVolumeSpecName "kube-api-access-pvl8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:07.569120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.569052    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:07.569120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.569081    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvl8n\" (UniqueName: \"kubernetes.io/projected/8327232a-4d38-4266-a5e8-dc464ea7fb0a-kube-api-access-pvl8n\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:07.569120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.569091    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8327232a-4d38-4266-a5e8-dc464ea7fb0a-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:07.569120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.569100    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8327232a-4d38-4266-a5e8-dc464ea7fb0a-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:07.706257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.706222    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"]
Oct 30 19:27:07.706732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.706702    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy" containerID="cri-o://ab50489bfac205d23356f178f0383b56a1d4c34cb33f9cb038a4f7faadd5b075" gracePeriod=30
Oct 30 19:27:07.839472 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.839439    5232 generic.go:356] "Generic (PLEG): container finished" podID="8327232a-4d38-4266-a5e8-dc464ea7fb0a" containerID="63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf" exitCode=0
Oct 30 19:27:07.839645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.839499    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"
Oct 30 19:27:07.839645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.839525    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl" event={"ID":"8327232a-4d38-4266-a5e8-dc464ea7fb0a","Type":"ContainerDied","Data":"63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf"}
Oct 30 19:27:07.839645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.839576    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl" event={"ID":"8327232a-4d38-4266-a5e8-dc464ea7fb0a","Type":"ContainerDied","Data":"7634f972eb8c310ed8631bd3046a6b71f3ac934f0f6c3dcc4f1761a828740a1b"}
Oct 30 19:27:07.839645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.839602    5232 scope.go:122] "RemoveContainer" containerID="63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf"
Oct 30 19:27:07.849703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.849683    5232 scope.go:122] "RemoveContainer" containerID="63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf"
Oct 30 19:27:07.850019 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:27:07.849992    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf\": container with ID starting with 63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf not found: ID does not exist" containerID="63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf"
Oct 30 19:27:07.850096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:07.850032    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf"} err="failed to get container status \"63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf\": rpc error: code = NotFound desc = could not find container \"63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf\": container with ID starting with 63a5cc00bdff9809431734ace7d823c941b9a3172358a91e0964c2025f502bcf not found: ID does not exist"
Oct 30 19:27:08.057103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:08.057068    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"]
Oct 30 19:27:08.059321 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:08.059284    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4ec347-7747f5cd75-qtldl"]
Oct 30 19:27:09.437372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:09.437335    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:27:09.437807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:09.437410    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:27:09.539480 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:09.539447    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8327232a-4d38-4266-a5e8-dc464ea7fb0a" path="/var/lib/kubelet/pods/8327232a-4d38-4266-a5e8-dc464ea7fb0a/volumes"
Oct 30 19:27:12.688007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.687941    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:27:12.845430 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845395    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-envoy\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.845651 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845600    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-certs\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.845743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845671    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istiod-ca-cert\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.845802 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845774    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-podinfo\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.845904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845845    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-credential-socket\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.845904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845879    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-socket\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.846008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845904    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-certs" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:12.846008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845938    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-kube-api-access-7xtkz\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-kube-api-access-7xtkz\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.846008 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.845974    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-data\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.846159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846036    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-token\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-token\") pod \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\" (UID: \"8dc2f14b-8fb0-4435-97eb-72440e29cdb9\") "
Oct 30 19:27:12.846438 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846317    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-socket" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:12.846438 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846338    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-data" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:12.846598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846471    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-credential-socket" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:12.846598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846554    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istiod-ca-cert" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:12.846693 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846608    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.846693 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846628    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.846693 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846642    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.846693 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.846656    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.848074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.848039    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-envoy" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:12.848283 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.848255    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-kube-api-access-7xtkz" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "kube-api-access-7xtkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:12.848728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.848464    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-podinfo" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:27:12.849584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.849559    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-token" pod "8dc2f14b-8fb0-4435-97eb-72440e29cdb9" (UID: "8dc2f14b-8fb0-4435-97eb-72440e29cdb9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:12.875862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.875806    5232 generic.go:356] "Generic (PLEG): container finished" podID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerID="d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505" exitCode=0
Oct 30 19:27:12.876030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.875881    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"
Oct 30 19:27:12.876030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.875937    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" event={"ID":"8dc2f14b-8fb0-4435-97eb-72440e29cdb9","Type":"ContainerDied","Data":"d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505"}
Oct 30 19:27:12.876030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.876007    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn" event={"ID":"8dc2f14b-8fb0-4435-97eb-72440e29cdb9","Type":"ContainerDied","Data":"6aeabb6662175ece39feb55d50ff9dd04b690b51f5cdd680de70f3ac5c323156"}
Oct 30 19:27:12.876187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.876040    5232 scope.go:122] "RemoveContainer" containerID="d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505"
Oct 30 19:27:12.878623 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.878590    5232 generic.go:356] "Generic (PLEG): container finished" podID="1c927693-f0c9-4831-930e-9db28fd12ace" containerID="ab50489bfac205d23356f178f0383b56a1d4c34cb33f9cb038a4f7faadd5b075" exitCode=0
Oct 30 19:27:12.878756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.878627    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" event={"ID":"1c927693-f0c9-4831-930e-9db28fd12ace","Type":"ContainerDied","Data":"ab50489bfac205d23356f178f0383b56a1d4c34cb33f9cb038a4f7faadd5b075"}
Oct 30 19:27:12.892310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.892085    5232 scope.go:122] "RemoveContainer" containerID="d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505"
Oct 30 19:27:12.899887 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:27:12.899433    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505\": container with ID starting with d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505 not found: ID does not exist" containerID="d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505"
Oct 30 19:27:12.899887 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.899480    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505"} err="failed to get container status \"d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505\": rpc error: code = NotFound desc = could not find container \"d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505\": container with ID starting with d1110d090cb5126ccef3d0e7352132d05004db114d111b1541f72e935aa50505 not found: ID does not exist"
Oct 30 19:27:12.899887 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.899505    5232 scope.go:122] "RemoveContainer" containerID="d71cdeb6d7b0be1032696a6c0467683c9ff557e35f0072bb78e64acdbc3f5c5d"
Oct 30 19:27:12.947508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.947428    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.947508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.947456    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.947508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.947468    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.947508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.947477    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.947508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.947490    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xtkz\" (UniqueName: \"kubernetes.io/projected/8dc2f14b-8fb0-4435-97eb-72440e29cdb9-kube-api-access-7xtkz\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:12.980051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:12.980026    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:27:13.038663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.038626    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"]
Oct 30 19:27:13.043380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.043351    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-qnltn"]
Oct 30 19:27:13.048310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048281    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c927693-f0c9-4831-930e-9db28fd12ace-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c927693-f0c9-4831-930e-9db28fd12ace-istiod-ca-cert\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.048449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048329    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-certs\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.048449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048365    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-credential-socket\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.048449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048428    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-data\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.048598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048472    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1c927693-f0c9-4831-930e-9db28fd12ace-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c927693-f0c9-4831-930e-9db28fd12ace-istio-podinfo\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.048598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048506    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-envoy\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.048598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048588    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-kube-api-access-t47fr\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-kube-api-access-t47fr\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.048756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048649    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-istio-token\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-istio-token\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.048756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048702    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-certs" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:13.048756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048722    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-credential-socket" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:13.048949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048789    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-data" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:13.048949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.048795    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-socket\") pod \"1c927693-f0c9-4831-930e-9db28fd12ace\" (UID: \"1c927693-f0c9-4831-930e-9db28fd12ace\") "
Oct 30 19:27:13.049048 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.049015    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-socket" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:13.049293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.049268    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c927693-f0c9-4831-930e-9db28fd12ace-istiod-ca-cert" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:13.049409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.049379    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.049409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.049392    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c927693-f0c9-4831-930e-9db28fd12ace-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.049409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.049403    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.049569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.049416    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.049569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.049430    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.051297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.051272    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c927693-f0c9-4831-930e-9db28fd12ace-istio-podinfo" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:27:13.051371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.051281    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-envoy" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:13.051371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.051301    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-istio-token" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:13.051371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.051357    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-kube-api-access-t47fr" pod "1c927693-f0c9-4831-930e-9db28fd12ace" (UID: "1c927693-f0c9-4831-930e-9db28fd12ace"). InnerVolumeSpecName "kube-api-access-t47fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:13.150726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.150688    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c927693-f0c9-4831-930e-9db28fd12ace-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.150726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.150721    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c927693-f0c9-4831-930e-9db28fd12ace-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.150967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.150734    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t47fr\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-kube-api-access-t47fr\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.150967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.150751    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c927693-f0c9-4831-930e-9db28fd12ace-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:13.541405 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.541323    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" path="/var/lib/kubelet/pods/8dc2f14b-8fb0-4435-97eb-72440e29cdb9/volumes"
Oct 30 19:27:13.887730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.887701    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"
Oct 30 19:27:13.888228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.887747    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk" event={"ID":"1c927693-f0c9-4831-930e-9db28fd12ace","Type":"ContainerDied","Data":"27ce1dd104496ec10662f651010c51b5f71550f5854f43212ea76b019015e7b4"}
Oct 30 19:27:13.888228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:13.887808    5232 scope.go:122] "RemoveContainer" containerID="ab50489bfac205d23356f178f0383b56a1d4c34cb33f9cb038a4f7faadd5b075"
Oct 30 19:27:14.047692 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:14.047660    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"]
Oct 30 19:27:14.095302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:14.095266    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-905566-openshift-default-7b9798f5d9-bvdsk"]
Oct 30 19:27:15.540645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:15.540605    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" path="/var/lib/kubelet/pods/1c927693-f0c9-4831-930e-9db28fd12ace/volumes"
Oct 30 19:27:16.190363 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:27:16.190316    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:27:32.226985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:32.226947    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:27:32.227511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:32.227037    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:27:34.072541 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:34.072507    5232 generic.go:356] "Generic (PLEG): container finished" podID="50030bcc-3a51-420a-8542-fde4481596db" containerID="111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396" exitCode=0
Oct 30 19:27:34.073107 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:34.072585    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" event={"ID":"50030bcc-3a51-420a-8542-fde4481596db","Type":"ContainerDied","Data":"111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396"}
Oct 30 19:27:34.073191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:34.073104    5232 scope.go:122] "RemoveContainer" containerID="111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396"
Oct 30 19:27:34.074495 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:34.074467    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:34.074580 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:34.074532    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:34.074580 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:34.074565    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:27:35.080882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:35.080843    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" event={"ID":"50030bcc-3a51-420a-8542-fde4481596db","Type":"ContainerStarted","Data":"7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8"}
Oct 30 19:27:35.671622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:35.671585    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:27:35.671871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:35.671847    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:27:35.671949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:35.671912    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:27:36.672155 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:36.672118    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:27:36.672593 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:36.672181    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:27:37.672363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:37.672316    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:27:37.672763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:37.672387    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:27:38.672357 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:38.672317    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:27:38.672535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:38.672369    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:27:39.614100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:39.614067    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"]
Oct 30 19:27:39.614288 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:39.614265    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy" containerID="cri-o://9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40" gracePeriod=30
Oct 30 19:27:39.671862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:39.671791    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:27:39.672030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:39.671892    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:27:40.672243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:40.672208    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:27:40.672706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:40.672268    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:27:41.676949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:41.676916    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:27:41.677387 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:41.677240    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:27:41.678081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:41.678064    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:27:44.873496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:44.873464    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:27:45.049576 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049495    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-certs\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.049721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049574    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-podinfo\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.049721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049620    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istiod-ca-cert\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.049721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049666    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-envoy\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.049721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049699    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-token\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-token\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.049963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049756    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-certs" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:45.049963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049802    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-credential-socket\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.049963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049879    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-data\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.049963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049947    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-kube-api-access-52f2n\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-kube-api-access-52f2n\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.050191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.049984    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-socket\") pod \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\" (UID: \"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4\") "
Oct 30 19:27:45.050191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.050094    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istiod-ca-cert" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:45.050361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.050325    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-socket" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:45.050542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.050517    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-data" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:45.050704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.050625    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.050871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.050848    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-credential-socket" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:45.051036 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.050863    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.051155 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.051046    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.052278 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.052254    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-envoy" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:45.052491 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.052468    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-token" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:45.052750 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.052730    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-kube-api-access-52f2n" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "kube-api-access-52f2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:45.052834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.052730    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-podinfo" pod "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" (UID: "dd9a17d9-50a6-4063-8ae9-9690fdfd4df4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:27:45.155465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.155426    5232 generic.go:356] "Generic (PLEG): container finished" podID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerID="9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40" exitCode=0
Oct 30 19:27:45.155765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.155740    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"
Oct 30 19:27:45.155952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.155926    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" event={"ID":"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4","Type":"ContainerDied","Data":"9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40"}
Oct 30 19:27:45.156152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.156116    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s" event={"ID":"dd9a17d9-50a6-4063-8ae9-9690fdfd4df4","Type":"ContainerDied","Data":"1f8d7d4e2dbbba74edbdcb0a8615f022449e5602d61f77dd00536d0cb2bcf031"}
Oct 30 19:27:45.156276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.156165    5232 scope.go:122] "RemoveContainer" containerID="9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40"
Oct 30 19:27:45.158301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.156678    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.158301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.156713    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.158301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.156729    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52f2n\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-kube-api-access-52f2n\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.158301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.156745    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.158301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.156760    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.158301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.156779    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:45.167495 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.167476    5232 scope.go:122] "RemoveContainer" containerID="9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40"
Oct 30 19:27:45.167765 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:27:45.167746    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40\": container with ID starting with 9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40 not found: ID does not exist" containerID="9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40"
Oct 30 19:27:45.167854 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.167774    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40"} err="failed to get container status \"9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40\": rpc error: code = NotFound desc = could not find container \"9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40\": container with ID starting with 9cce85571ace7edc2450177516ed191d12ec751645c179efdd53f3f37f2f9e40 not found: ID does not exist"
Oct 30 19:27:45.317433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.317348    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"]
Oct 30 19:27:45.324465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.324432    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-7wr8s"]
Oct 30 19:27:45.539631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:27:45.539582    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" path="/var/lib/kubelet/pods/dd9a17d9-50a6-4063-8ae9-9690fdfd4df4/volumes"
Oct 30 19:27:46.186827 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:27:46.186776    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:28:03.046157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.046118    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"]
Oct 30 19:28:03.046985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.046959    5232 cpu_manager.go:414] "containerMap: removing container" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy"
Oct 30 19:28:03.046985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.046984    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047004    5232 cpu_manager.go:414] "containerMap: removing container" podUID="90484b62-de48-4b7a-8e3b-da5c11f9862d" containerName="discovery"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047012    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90484b62-de48-4b7a-8e3b-da5c11f9862d" containerName="discovery"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047034    5232 cpu_manager.go:414] "containerMap: removing container" podUID="8327232a-4d38-4266-a5e8-dc464ea7fb0a" containerName="maas-api"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047044    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8327232a-4d38-4266-a5e8-dc464ea7fb0a" containerName="maas-api"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047068    5232 cpu_manager.go:414] "containerMap: removing container" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047075    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047104    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047111    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047122    5232 cpu_manager.go:414] "containerMap: removing container" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047130    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047158    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy"
Oct 30 19:28:03.047162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047164    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy"
Oct 30 19:28:03.047728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047285    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="90484b62-de48-4b7a-8e3b-da5c11f9862d" containerName="discovery"
Oct 30 19:28:03.047728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047302    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="8327232a-4d38-4266-a5e8-dc464ea7fb0a" containerName="maas-api"
Oct 30 19:28:03.047728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047313    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="8dc2f14b-8fb0-4435-97eb-72440e29cdb9" containerName="istio-proxy"
Oct 30 19:28:03.047728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047326    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy"
Oct 30 19:28:03.047728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047334    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd9a17d9-50a6-4063-8ae9-9690fdfd4df4" containerName="istio-proxy"
Oct 30 19:28:03.047728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047345    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy"
Oct 30 19:28:03.047728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047356    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c927693-f0c9-4831-930e-9db28fd12ace" containerName="istio-proxy"
Oct 30 19:28:03.047728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.047365    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy"
Oct 30 19:28:03.052975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.052951    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.104641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.104605    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-c16ba7-openshift-default-dockercfg-524lv\""
Oct 30 19:28:03.154896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.154858    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"]
Oct 30 19:28:03.165316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165278    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnwl5\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-kube-api-access-rnwl5\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.165505 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165341    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istiod-ca-cert\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.165505 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165447    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-data\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.165505 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165475    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-podinfo\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.165681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165595    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-credential-socket\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.165681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165656    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-token\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.165765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165741    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-envoy\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.165803 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165780    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-socket\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.165894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.165803    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-certs\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.266919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.266882    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-token\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.266943    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-envoy\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.266966    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-socket\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.266991    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-certs\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267058    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnwl5\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-kube-api-access-rnwl5\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267096    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istiod-ca-cert\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267125    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-data\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267153    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-podinfo\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267226    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-credential-socket\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267423    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-socket\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267686 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267631    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-certs\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267686 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267666    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-credential-socket\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267716    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-data\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.267878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.267748    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istiod-ca-cert\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.269388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.269365    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-envoy\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.269738 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.269708    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-podinfo\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.339701 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.339668    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnwl5\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-kube-api-access-rnwl5\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.351830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.351784    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-token\") pod \"e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") " pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.368115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.368082    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:03.801180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.801059    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"]
Oct 30 19:28:03.805777 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:28:03.805751    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68859be2_cf72_46a2_bc8a_5d7ba22c0a4c.slice/crio-fa621c5db0d7d63a35bab7d6831ed8f80c1b37e6ba0369a6333eed8a5bb61885 WatchSource:0}: Error finding container fa621c5db0d7d63a35bab7d6831ed8f80c1b37e6ba0369a6333eed8a5bb61885: Status 404 returned error can't find the container with id fa621c5db0d7d63a35bab7d6831ed8f80c1b37e6ba0369a6333eed8a5bb61885
Oct 30 19:28:03.808232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.808198    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:03.808313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.808271    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:03.808313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:03.808305    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:04.276101 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:04.276065    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" event={"ID":"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c","Type":"ContainerStarted","Data":"90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb"}
Oct 30 19:28:04.276644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:04.276129    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" event={"ID":"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c","Type":"ContainerStarted","Data":"fa621c5db0d7d63a35bab7d6831ed8f80c1b37e6ba0369a6333eed8a5bb61885"}
Oct 30 19:28:04.368897 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:04.368853    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:04.374665 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:04.374636    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:04.395781 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:04.395720    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" podStartSLOduration=2.395704237 podStartE2EDuration="2.395704237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:04.393918404 +0000 UTC m=+2476.978401405" watchObservedRunningTime="2026-10-30 19:28:04.395704237 +0000 UTC m=+2476.980187234"
Oct 30 19:28:05.282537 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:05.282503    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:05.283626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:05.283606    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:28:12.734885 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:12.734852    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"]
Oct 30 19:28:12.735473 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:12.735046    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w" podUID="a4e80b7f-f318-4f87-9c51-856b871be5c6" containerName="maas-api" containerID="cri-o://a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7" gracePeriod=30
Oct 30 19:28:12.857135 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:12.857100    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w" podUID="a4e80b7f-f318-4f87-9c51-856b871be5c6" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.122:8443/health\": dial tcp 10.133.0.122:8443: connect: connection refused"
Oct 30 19:28:13.011128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.011101    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:28:13.187075 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.187038    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls\") pod \"a4e80b7f-f318-4f87-9c51-856b871be5c6\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") "
Oct 30 19:28:13.187242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.187153    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4e80b7f-f318-4f87-9c51-856b871be5c6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e80b7f-f318-4f87-9c51-856b871be5c6-openshift-service-ca\") pod \"a4e80b7f-f318-4f87-9c51-856b871be5c6\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") "
Oct 30 19:28:13.187242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.187227    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-metrics-certs\") pod \"a4e80b7f-f318-4f87-9c51-856b871be5c6\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") "
Oct 30 19:28:13.187320 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.187252    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4e80b7f-f318-4f87-9c51-856b871be5c6-kube-api-access-jfhfj\" (UniqueName: \"kubernetes.io/projected/a4e80b7f-f318-4f87-9c51-856b871be5c6-kube-api-access-jfhfj\") pod \"a4e80b7f-f318-4f87-9c51-856b871be5c6\" (UID: \"a4e80b7f-f318-4f87-9c51-856b871be5c6\") "
Oct 30 19:28:13.187502 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.187473    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4e80b7f-f318-4f87-9c51-856b871be5c6-openshift-service-ca" pod "a4e80b7f-f318-4f87-9c51-856b871be5c6" (UID: "a4e80b7f-f318-4f87-9c51-856b871be5c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:13.187636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.187603    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4e80b7f-f318-4f87-9c51-856b871be5c6-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:13.189293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.189262    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-metrics-certs" pod "a4e80b7f-f318-4f87-9c51-856b871be5c6" (UID: "a4e80b7f-f318-4f87-9c51-856b871be5c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:13.189383 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.189295    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4e80b7f-f318-4f87-9c51-856b871be5c6-kube-api-access-jfhfj" pod "a4e80b7f-f318-4f87-9c51-856b871be5c6" (UID: "a4e80b7f-f318-4f87-9c51-856b871be5c6"). InnerVolumeSpecName "kube-api-access-jfhfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:13.189427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.189381    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls" pod "a4e80b7f-f318-4f87-9c51-856b871be5c6" (UID: "a4e80b7f-f318-4f87-9c51-856b871be5c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:13.288450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.288378    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:13.288450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.288407    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfhfj\" (UniqueName: \"kubernetes.io/projected/a4e80b7f-f318-4f87-9c51-856b871be5c6-kube-api-access-jfhfj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:13.288450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.288417    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4e80b7f-f318-4f87-9c51-856b871be5c6-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:13.330888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.330852    5232 generic.go:356] "Generic (PLEG): container finished" podID="a4e80b7f-f318-4f87-9c51-856b871be5c6" containerID="a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7" exitCode=0
Oct 30 19:28:13.331041 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.330899    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w" event={"ID":"a4e80b7f-f318-4f87-9c51-856b871be5c6","Type":"ContainerDied","Data":"a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7"}
Oct 30 19:28:13.331041 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.330940    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w" event={"ID":"a4e80b7f-f318-4f87-9c51-856b871be5c6","Type":"ContainerDied","Data":"19e8422a35446b52196502b5984be9aa0f71240832f0bcd3917572e96cb4c9c5"}
Oct 30 19:28:13.331041 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.330961    5232 scope.go:122] "RemoveContainer" containerID="a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7"
Oct 30 19:28:13.331041 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.330902    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"
Oct 30 19:28:13.341525 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.341505    5232 scope.go:122] "RemoveContainer" containerID="a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7"
Oct 30 19:28:13.341777 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:28:13.341758    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7\": container with ID starting with a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7 not found: ID does not exist" containerID="a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7"
Oct 30 19:28:13.341871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.341783    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7"} err="failed to get container status \"a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7\": rpc error: code = NotFound desc = could not find container \"a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7\": container with ID starting with a23c709f6c10d3128308e85280acab020c341e4c4cf22ab9fed81d89215f21b7 not found: ID does not exist"
Oct 30 19:28:13.490734 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.490698    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"]
Oct 30 19:28:13.501609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.497079    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-539084-866dfb6cd5-6xv7w"]
Oct 30 19:28:13.539195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:13.539126    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4e80b7f-f318-4f87-9c51-856b871be5c6" path="/var/lib/kubelet/pods/a4e80b7f-f318-4f87-9c51-856b871be5c6/volumes"
Oct 30 19:28:16.186967 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:28:16.186929    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:28:23.176699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:23.176661    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"]
Oct 30 19:28:23.177258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:23.176930    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy" containerID="cri-o://102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd" gracePeriod=30
Oct 30 19:28:25.629585 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.629551    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"]
Oct 30 19:28:25.630145 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.630127    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a4e80b7f-f318-4f87-9c51-856b871be5c6" containerName="maas-api"
Oct 30 19:28:25.630197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.630148    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4e80b7f-f318-4f87-9c51-856b871be5c6" containerName="maas-api"
Oct 30 19:28:25.630197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.630170    5232 cpu_manager.go:414] "containerMap: removing container" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy"
Oct 30 19:28:25.630197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.630176    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="372dccb4-44bc-44c8-8d0b-819378f3f085" containerName="istio-proxy"
Oct 30 19:28:25.630291 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.630269    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4e80b7f-f318-4f87-9c51-856b871be5c6" containerName="maas-api"
Oct 30 19:28:25.634939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.634919    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.685526 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.685485    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-f9a20d-openshift-default-dockercfg-hkk47\""
Oct 30 19:28:25.734231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734181    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdxdl\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-kube-api-access-xdxdl\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.734231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734230    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-token\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.734488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734277    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-socket\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.734488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734320    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc314899-ffa0-49eb-a97b-1bde15e33c58-istiod-ca-cert\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.734488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734370    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-credential-socket\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.734488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734430    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-podinfo\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.734488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734468    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-certs\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.734787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734511    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-data\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.734787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.734567    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-envoy\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.737257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.737227    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"]
Oct 30 19:28:25.737504 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.737472    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs" podUID="81c864f6-81b6-4d20-8f9a-600df2ad752b" containerName="maas-api" containerID="cri-o://0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb" gracePeriod=30
Oct 30 19:28:25.743067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.743013    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"]
Oct 30 19:28:25.835990 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.835954    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-socket\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836018    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc314899-ffa0-49eb-a97b-1bde15e33c58-istiod-ca-cert\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836068    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-credential-socket\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836119    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-podinfo\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836155    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-certs\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836464 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836429    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-socket\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836481    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-data\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836487    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-certs\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836612 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836549    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-envoy\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836612 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836562    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-credential-socket\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836725 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836692    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-data\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836725 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836706    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdxdl\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-kube-api-access-xdxdl\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836869 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836734    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-token\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.836869 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.836845    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc314899-ffa0-49eb-a97b-1bde15e33c58-istiod-ca-cert\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.838713 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.838690    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-envoy\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.838866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.838771    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-podinfo\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.896860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.896735    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-token\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.897032 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.896886    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdxdl\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-kube-api-access-xdxdl\") pod \"e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") " pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.948048 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.948012    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:25.999986 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:25.999957    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:28:26.139392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.139355    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81c864f6-81b6-4d20-8f9a-600df2ad752b-kube-api-access-dx2rt\" (UniqueName: \"kubernetes.io/projected/81c864f6-81b6-4d20-8f9a-600df2ad752b-kube-api-access-dx2rt\") pod \"81c864f6-81b6-4d20-8f9a-600df2ad752b\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") "
Oct 30 19:28:26.139595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.139457    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls\") pod \"81c864f6-81b6-4d20-8f9a-600df2ad752b\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") "
Oct 30 19:28:26.139673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.139604    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81c864f6-81b6-4d20-8f9a-600df2ad752b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c864f6-81b6-4d20-8f9a-600df2ad752b-openshift-service-ca\") pod \"81c864f6-81b6-4d20-8f9a-600df2ad752b\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") "
Oct 30 19:28:26.139673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.139643    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-metrics-certs\") pod \"81c864f6-81b6-4d20-8f9a-600df2ad752b\" (UID: \"81c864f6-81b6-4d20-8f9a-600df2ad752b\") "
Oct 30 19:28:26.140550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.139985    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81c864f6-81b6-4d20-8f9a-600df2ad752b-openshift-service-ca" pod "81c864f6-81b6-4d20-8f9a-600df2ad752b" (UID: "81c864f6-81b6-4d20-8f9a-600df2ad752b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:26.140550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.140112    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c864f6-81b6-4d20-8f9a-600df2ad752b-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:26.142194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.142170    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c864f6-81b6-4d20-8f9a-600df2ad752b-kube-api-access-dx2rt" pod "81c864f6-81b6-4d20-8f9a-600df2ad752b" (UID: "81c864f6-81b6-4d20-8f9a-600df2ad752b"). InnerVolumeSpecName "kube-api-access-dx2rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:26.142448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.142428    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-metrics-certs" pod "81c864f6-81b6-4d20-8f9a-600df2ad752b" (UID: "81c864f6-81b6-4d20-8f9a-600df2ad752b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:26.145612 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.145585    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls" pod "81c864f6-81b6-4d20-8f9a-600df2ad752b" (UID: "81c864f6-81b6-4d20-8f9a-600df2ad752b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:26.241511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.241428    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:26.241511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.241463    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx2rt\" (UniqueName: \"kubernetes.io/projected/81c864f6-81b6-4d20-8f9a-600df2ad752b-kube-api-access-dx2rt\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:26.241511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.241474    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c864f6-81b6-4d20-8f9a-600df2ad752b-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:26.384229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.384194    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"]
Oct 30 19:28:26.388435 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:28:26.388407    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc314899_ffa0_49eb_a97b_1bde15e33c58.slice/crio-afadadb58b0cf1f0b2df4e669a37aa3d4f1da97cf942ee8cf54ca22a72905cf0 WatchSource:0}: Error finding container afadadb58b0cf1f0b2df4e669a37aa3d4f1da97cf942ee8cf54ca22a72905cf0: Status 404 returned error can't find the container with id afadadb58b0cf1f0b2df4e669a37aa3d4f1da97cf942ee8cf54ca22a72905cf0
Oct 30 19:28:26.391212 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.391174    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:26.391287 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.391261    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:26.391325 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.391304    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:26.419589 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.419556    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" event={"ID":"fc314899-ffa0-49eb-a97b-1bde15e33c58","Type":"ContainerStarted","Data":"afadadb58b0cf1f0b2df4e669a37aa3d4f1da97cf942ee8cf54ca22a72905cf0"}
Oct 30 19:28:26.421428 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.421400    5232 generic.go:356] "Generic (PLEG): container finished" podID="81c864f6-81b6-4d20-8f9a-600df2ad752b" containerID="0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb" exitCode=0
Oct 30 19:28:26.421564 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.421452    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"
Oct 30 19:28:26.421622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.421563    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs" event={"ID":"81c864f6-81b6-4d20-8f9a-600df2ad752b","Type":"ContainerDied","Data":"0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb"}
Oct 30 19:28:26.421622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.421607    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs" event={"ID":"81c864f6-81b6-4d20-8f9a-600df2ad752b","Type":"ContainerDied","Data":"5bc2f940d88d04798a7bb7403f2a2bbd212c1141d66de6e650a102d659628ac1"}
Oct 30 19:28:26.421721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.421638    5232 scope.go:122] "RemoveContainer" containerID="0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb"
Oct 30 19:28:26.448070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.448045    5232 scope.go:122] "RemoveContainer" containerID="0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb"
Oct 30 19:28:26.448412 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:28:26.448391    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb\": container with ID starting with 0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb not found: ID does not exist" containerID="0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb"
Oct 30 19:28:26.448600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.448421    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb"} err="failed to get container status \"0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb\": rpc error: code = NotFound desc = could not find container \"0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb\": container with ID starting with 0b2bb3a233eae92c17cbf402ae72d83d6585a25a12c0c39c76f483b30fa91ebb not found: ID does not exist"
Oct 30 19:28:26.600434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.600401    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"]
Oct 30 19:28:26.616087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:26.616054    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-496c5b-df65f7967-c62cs"]
Oct 30 19:28:27.429603 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:27.429561    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" event={"ID":"fc314899-ffa0-49eb-a97b-1bde15e33c58","Type":"ContainerStarted","Data":"ededd401189e94debc6ed3d260571c9ecd4706a434c96977ad4d169b8815f2aa"}
Oct 30 19:28:27.539488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:27.539452    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81c864f6-81b6-4d20-8f9a-600df2ad752b" path="/var/lib/kubelet/pods/81c864f6-81b6-4d20-8f9a-600df2ad752b/volumes"
Oct 30 19:28:27.550350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:27.550296    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" podStartSLOduration=2.5502787749999998 podStartE2EDuration="2.550278775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:27.547721764 +0000 UTC m=+2500.132204762" watchObservedRunningTime="2026-10-30 19:28:27.550278775 +0000 UTC m=+2500.134761775"
Oct 30 19:28:27.948992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:27.948955    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:27.954591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:27.954565    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:28.438459 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.438434    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:28:28.439490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.439465    5232 generic.go:356] "Generic (PLEG): container finished" podID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerID="102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd" exitCode=0
Oct 30 19:28:28.439609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.439543    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" event={"ID":"d60cfa3f-582b-4634-b75a-99344533b5b6","Type":"ContainerDied","Data":"102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd"}
Oct 30 19:28:28.439660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.439610    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw" event={"ID":"d60cfa3f-582b-4634-b75a-99344533b5b6","Type":"ContainerDied","Data":"1534cfb98137f3ab3d1b9bba5cfd46c0d67eb03bebdd06144cb49dba18bcc4fa"}
Oct 30 19:28:28.439660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.439646    5232 scope.go:122] "RemoveContainer" containerID="102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd"
Oct 30 19:28:28.439985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.439962    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:28.440930 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.440913    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:28:28.450895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.450880    5232 scope.go:122] "RemoveContainer" containerID="0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2"
Oct 30 19:28:28.459710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.459691    5232 scope.go:122] "RemoveContainer" containerID="102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd"
Oct 30 19:28:28.460008 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:28:28.459987    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd\": container with ID starting with 102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd not found: ID does not exist" containerID="102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd"
Oct 30 19:28:28.460050 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.460018    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd"} err="failed to get container status \"102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd\": rpc error: code = NotFound desc = could not find container \"102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd\": container with ID starting with 102568b6761bf7366d8c49b249d6fed417d94ab6a0281181dfdaf114920e5abd not found: ID does not exist"
Oct 30 19:28:28.460050 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.460036    5232 scope.go:122] "RemoveContainer" containerID="0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2"
Oct 30 19:28:28.460287 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:28:28.460266    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2\": container with ID starting with 0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2 not found: ID does not exist" containerID="0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2"
Oct 30 19:28:28.460343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.460297    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2"} err="failed to get container status \"0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2\": rpc error: code = NotFound desc = could not find container \"0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2\": container with ID starting with 0de17f7a652e741c3ab09c46b34e293a22edc83d4ccef2ffc62aeef94988f3d2 not found: ID does not exist"
Oct 30 19:28:28.568988 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.568909    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-envoy\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.569175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569023    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-podinfo\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.569175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569053    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-token\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-token\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.569175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569110    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-kube-api-access-9jh4g\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-kube-api-access-9jh4g\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.569175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569141    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-credential-socket\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.569411 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569190    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-certs\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.569411 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569228    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-data\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.569411 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569283    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d60cfa3f-582b-4634-b75a-99344533b5b6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d60cfa3f-582b-4634-b75a-99344533b5b6-istiod-ca-cert\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.569411 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569333    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-socket\") pod \"d60cfa3f-582b-4634-b75a-99344533b5b6\" (UID: \"d60cfa3f-582b-4634-b75a-99344533b5b6\") "
Oct 30 19:28:28.570339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569709    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-socket" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:28:28.570339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569747    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-certs" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:28:28.570339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.569853    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-data" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:28:28.570339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.570059    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-credential-socket" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:28:28.570339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.570106    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d60cfa3f-582b-4634-b75a-99344533b5b6-istiod-ca-cert" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:28.570760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.570421    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d60cfa3f-582b-4634-b75a-99344533b5b6-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:28.570760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.570441    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:28.570760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.570455    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:28.570760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.570494    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:28.570760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.570508    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:28.572272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.572247    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-podinfo" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:28:28.572496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.572470    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-envoy" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:28:28.572870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.572844    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-token" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:28.573037 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.572898    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-kube-api-access-9jh4g" pod "d60cfa3f-582b-4634-b75a-99344533b5b6" (UID: "d60cfa3f-582b-4634-b75a-99344533b5b6"). InnerVolumeSpecName "kube-api-access-9jh4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:28.671452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.671417    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jh4g\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-kube-api-access-9jh4g\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:28.671452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.671445    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:28.671452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.671456    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:28.671696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:28.671466    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d60cfa3f-582b-4634-b75a-99344533b5b6-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:29.446368 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:29.446335    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"
Oct 30 19:28:29.621910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:29.621858    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"]
Oct 30 19:28:29.669192 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:29.669151    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-539084-openshift-default-64588b5875-rgfvw"]
Oct 30 19:28:31.539960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:31.539925    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" path="/var/lib/kubelet/pods/d60cfa3f-582b-4634-b75a-99344533b5b6/volumes"
Oct 30 19:28:34.567566 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.567531    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"]
Oct 30 19:28:34.568173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568154    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy"
Oct 30 19:28:34.568173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568174    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy"
Oct 30 19:28:34.568272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568186    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy"
Oct 30 19:28:34.568272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568191    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy"
Oct 30 19:28:34.568272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568217    5232 cpu_manager.go:414] "containerMap: removing container" podUID="81c864f6-81b6-4d20-8f9a-600df2ad752b" containerName="maas-api"
Oct 30 19:28:34.568272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568222    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81c864f6-81b6-4d20-8f9a-600df2ad752b" containerName="maas-api"
Oct 30 19:28:34.568425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568305    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="81c864f6-81b6-4d20-8f9a-600df2ad752b" containerName="maas-api"
Oct 30 19:28:34.568425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568317    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy"
Oct 30 19:28:34.568425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.568325    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d60cfa3f-582b-4634-b75a-99344533b5b6" containerName="istio-proxy"
Oct 30 19:28:34.573264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.573241    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:34.625307 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.625262    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-f9a20d-dockercfg-ftx2j\""
Oct 30 19:28:34.625515 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.625330    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-f9a20d\""
Oct 30 19:28:34.662050 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.662007    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxzdj\" (UniqueName: \"kubernetes.io/projected/acf82f67-3d24-460e-b7bd-a0b378b7452d-kube-api-access-wxzdj\") pod \"payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8\" (UID: \"acf82f67-3d24-460e-b7bd-a0b378b7452d\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:34.662886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.662860    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf82f67-3d24-460e-b7bd-a0b378b7452d-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8\" (UID: \"acf82f67-3d24-460e-b7bd-a0b378b7452d\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:34.687791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.686559    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"]
Oct 30 19:28:34.736103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.736071    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"]
Oct 30 19:28:34.740315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.740294    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:34.764485 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.764460    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxzdj\" (UniqueName: \"kubernetes.io/projected/acf82f67-3d24-460e-b7bd-a0b378b7452d-kube-api-access-wxzdj\") pod \"payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8\" (UID: \"acf82f67-3d24-460e-b7bd-a0b378b7452d\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:34.764628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.764495    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf82f67-3d24-460e-b7bd-a0b378b7452d-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8\" (UID: \"acf82f67-3d24-460e-b7bd-a0b378b7452d\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:34.765461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.765435    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf82f67-3d24-460e-b7bd-a0b378b7452d-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8\" (UID: \"acf82f67-3d24-460e-b7bd-a0b378b7452d\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:34.816665 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.816625    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"]
Oct 30 19:28:34.834046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.833970    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxzdj\" (UniqueName: \"kubernetes.io/projected/acf82f67-3d24-460e-b7bd-a0b378b7452d-kube-api-access-wxzdj\") pod \"payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8\" (UID: \"acf82f67-3d24-460e-b7bd-a0b378b7452d\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:34.866053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.866017    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnlkt\" (UniqueName: \"kubernetes.io/projected/f0609fae-2afd-4751-9f61-0fb155029862-kube-api-access-mnlkt\") pod \"payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9\" (UID: \"f0609fae-2afd-4751-9f61-0fb155029862\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:34.866216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.866080    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0609fae-2afd-4751-9f61-0fb155029862-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9\" (UID: \"f0609fae-2afd-4751-9f61-0fb155029862\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:34.886780 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.886751    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:34.967450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.967414    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnlkt\" (UniqueName: \"kubernetes.io/projected/f0609fae-2afd-4751-9f61-0fb155029862-kube-api-access-mnlkt\") pod \"payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9\" (UID: \"f0609fae-2afd-4751-9f61-0fb155029862\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:34.967602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.967508    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0609fae-2afd-4751-9f61-0fb155029862-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9\" (UID: \"f0609fae-2afd-4751-9f61-0fb155029862\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:34.968638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:34.968582    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0609fae-2afd-4751-9f61-0fb155029862-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9\" (UID: \"f0609fae-2afd-4751-9f61-0fb155029862\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:35.027873 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.027802    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnlkt\" (UniqueName: \"kubernetes.io/projected/f0609fae-2afd-4751-9f61-0fb155029862-kube-api-access-mnlkt\") pod \"payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9\" (UID: \"f0609fae-2afd-4751-9f61-0fb155029862\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:35.054582 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.054558    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:35.263774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.263745    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"]
Oct 30 19:28:35.267478 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:28:35.267449    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf82f67_3d24_460e_b7bd_a0b378b7452d.slice/crio-018848cc6a4bf33f70de4d0f52c99c58f0345cfeb8fde6448da6ccbbf9c61ad8 WatchSource:0}: Error finding container 018848cc6a4bf33f70de4d0f52c99c58f0345cfeb8fde6448da6ccbbf9c61ad8: Status 404 returned error can't find the container with id 018848cc6a4bf33f70de4d0f52c99c58f0345cfeb8fde6448da6ccbbf9c61ad8
Oct 30 19:28:35.435622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.435537    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"]
Oct 30 19:28:35.440504 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:28:35.440480    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0609fae_2afd_4751_9f61_0fb155029862.slice/crio-cff53809a7be5080ce154695aa8a353e4d5edc83563a19689e42d300b785becc WatchSource:0}: Error finding container cff53809a7be5080ce154695aa8a353e4d5edc83563a19689e42d300b785becc: Status 404 returned error can't find the container with id cff53809a7be5080ce154695aa8a353e4d5edc83563a19689e42d300b785becc
Oct 30 19:28:35.484886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.484850    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9" event={"ID":"f0609fae-2afd-4751-9f61-0fb155029862","Type":"ContainerStarted","Data":"cff53809a7be5080ce154695aa8a353e4d5edc83563a19689e42d300b785becc"}
Oct 30 19:28:35.486497 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.486473    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8" event={"ID":"acf82f67-3d24-460e-b7bd-a0b378b7452d","Type":"ContainerStarted","Data":"cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c"}
Oct 30 19:28:35.486604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.486524    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:35.486604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.486545    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8" event={"ID":"acf82f67-3d24-460e-b7bd-a0b378b7452d","Type":"ContainerStarted","Data":"018848cc6a4bf33f70de4d0f52c99c58f0345cfeb8fde6448da6ccbbf9c61ad8"}
Oct 30 19:28:35.603379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:35.603318    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8" podStartSLOduration=1.603300752 podStartE2EDuration="1.603300752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:35.600205368 +0000 UTC m=+2508.184688364" watchObservedRunningTime="2026-10-30 19:28:35.603300752 +0000 UTC m=+2508.187783749"
Oct 30 19:28:36.495272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:36.495229    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9" event={"ID":"f0609fae-2afd-4751-9f61-0fb155029862","Type":"ContainerStarted","Data":"289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7"}
Oct 30 19:28:36.495445 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:36.495314    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:36.613189 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:36.613135    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9" podStartSLOduration=2.613119493 podStartE2EDuration="2.613119493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:36.609558442 +0000 UTC m=+2509.194041437" watchObservedRunningTime="2026-10-30 19:28:36.613119493 +0000 UTC m=+2509.197602489"
Oct 30 19:28:42.090589 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.090554    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"]
Oct 30 19:28:42.094764 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.094746    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.182914 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.182873    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-441836-openshift-default-dockercfg-9l9wg\""
Oct 30 19:28:42.200884 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.200851    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"]
Oct 30 19:28:42.287628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.287580    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a453db-4937-4dab-8254-c2decafd1fb8-istio-podinfo\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.288017 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.287992    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-credential-socket\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.288247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.288217    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-istio-token\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.288427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.288411    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch4g5\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-kube-api-access-ch4g5\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.288600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.288586    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-socket\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.288752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.288738    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-certs\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.295868 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.288903    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-envoy\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.297219 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.297184    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a453db-4937-4dab-8254-c2decafd1fb8-istiod-ca-cert\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.297749 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.297727    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-data\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399326 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399231    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-istio-token\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399326 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399310    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch4g5\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-kube-api-access-ch4g5\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399554 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399357    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-socket\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399554 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399397    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-certs\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399554 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399424    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-envoy\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399554 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399526    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a453db-4937-4dab-8254-c2decafd1fb8-istiod-ca-cert\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399772 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399596    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-data\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399772 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399648    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a453db-4937-4dab-8254-c2decafd1fb8-istio-podinfo\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399772 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399686    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-credential-socket\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.399968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399884    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-socket\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.400028 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.399968    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-certs\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.400390 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.400363    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a453db-4937-4dab-8254-c2decafd1fb8-istiod-ca-cert\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.400471 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.400435    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-credential-socket\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.400558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.400535    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-data\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.401935 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.401912    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-envoy\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.402430 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.402407    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a453db-4937-4dab-8254-c2decafd1fb8-istio-podinfo\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.463378 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.463347    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-istio-token\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.463551 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.463526    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch4g5\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-kube-api-access-ch4g5\") pod \"e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") " pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:42.708895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:42.708789    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:43.202144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:43.202111    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"]
Oct 30 19:28:43.241969 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:28:43.241934    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a453db_4937_4dab_8254_c2decafd1fb8.slice/crio-be61673f595004a02ad7fa6bff7ec890f5e458c45c12cf130cb47f0ad2c4d319 WatchSource:0}: Error finding container be61673f595004a02ad7fa6bff7ec890f5e458c45c12cf130cb47f0ad2c4d319: Status 404 returned error can't find the container with id be61673f595004a02ad7fa6bff7ec890f5e458c45c12cf130cb47f0ad2c4d319
Oct 30 19:28:43.244501 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:43.244462    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:43.244590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:43.244539    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:43.244590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:43.244573    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:43.543501 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:43.543408    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" event={"ID":"73a453db-4937-4dab-8254-c2decafd1fb8","Type":"ContainerStarted","Data":"fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f"}
Oct 30 19:28:43.543501 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:43.543468    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" event={"ID":"73a453db-4937-4dab-8254-c2decafd1fb8","Type":"ContainerStarted","Data":"be61673f595004a02ad7fa6bff7ec890f5e458c45c12cf130cb47f0ad2c4d319"}
Oct 30 19:28:43.667451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:43.667387    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" podStartSLOduration=1.667367271 podStartE2EDuration="1.667367271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:43.663581013 +0000 UTC m=+2516.248064010" watchObservedRunningTime="2026-10-30 19:28:43.667367271 +0000 UTC m=+2516.251850272"
Oct 30 19:28:43.709903 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:43.709868    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:44.719555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:44.719522    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:45.559643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:45.559608    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:45.560705 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:45.560685    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:28:46.187588 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:28:46.187553    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:28:46.501636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:46.501539    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:28:47.504709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:47.504678    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:28:57.134265 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.134169    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"]
Oct 30 19:28:57.140547 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.140523    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.199329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.193504    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-f3b5af-openshift-default-dockercfg-x62l9\""
Oct 30 19:28:57.242932 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.242898    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"]
Oct 30 19:28:57.294538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.294503    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-istio-token\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.294744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.294561    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d1871f-405c-4db4-a424-4662b1e910cf-istiod-ca-cert\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.294744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.294644    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-envoy\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.294915 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.294764    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-credential-socket\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.294915 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.294846    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d1871f-405c-4db4-a424-4662b1e910cf-istio-podinfo\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.295027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.294962    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-socket\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.295027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.294998    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-certs\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.295139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.295115    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g9lj\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-kube-api-access-9g9lj\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.295198 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.295155    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-data\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398165 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398063    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g9lj\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-kube-api-access-9g9lj\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398165 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398114    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-data\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398165 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398143    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-istio-token\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398372    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d1871f-405c-4db4-a424-4662b1e910cf-istiod-ca-cert\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398672 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398539    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-envoy\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398672 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398663    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-data\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398822 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398661    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-credential-socket\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398822 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398739    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d1871f-405c-4db4-a424-4662b1e910cf-istio-podinfo\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398917 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398897    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-socket\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.398995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.398967    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-certs\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.399185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.399107    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-credential-socket\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.399322 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.399287    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-certs\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.399414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.399320    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d1871f-405c-4db4-a424-4662b1e910cf-istiod-ca-cert\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.399622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.399597    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-socket\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.402098 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.402073    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-envoy\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.402273 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.402142    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d1871f-405c-4db4-a424-4662b1e910cf-istio-podinfo\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.457774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.457743    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-istio-token\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.458419 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.458399    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g9lj\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-kube-api-access-9g9lj\") pod \"e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") " pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:57.753747 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:57.753669    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:58.190523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:58.190481    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"]
Oct 30 19:28:58.194905 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:28:58.194879    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d1871f_405c_4db4_a424_4662b1e910cf.slice/crio-162a0c1adc35d48692897adea961036f463c071b117dc7e9daa7eeb72bd89dd7 WatchSource:0}: Error finding container 162a0c1adc35d48692897adea961036f463c071b117dc7e9daa7eeb72bd89dd7: Status 404 returned error can't find the container with id 162a0c1adc35d48692897adea961036f463c071b117dc7e9daa7eeb72bd89dd7
Oct 30 19:28:58.197343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:58.197303    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:58.197462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:58.197410    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:58.197462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:58.197456    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:28:58.643263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:58.643220    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" event={"ID":"43d1871f-405c-4db4-a424-4662b1e910cf","Type":"ContainerStarted","Data":"fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2"}
Oct 30 19:28:58.643434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:58.643275    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" event={"ID":"43d1871f-405c-4db4-a424-4662b1e910cf","Type":"ContainerStarted","Data":"162a0c1adc35d48692897adea961036f463c071b117dc7e9daa7eeb72bd89dd7"}
Oct 30 19:28:58.754236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:58.754209    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:28:58.828242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:58.828184    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podStartSLOduration=1.828166577 podStartE2EDuration="1.828166577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:58.824760353 +0000 UTC m=+2531.409243352" watchObservedRunningTime="2026-10-30 19:28:58.828166577 +0000 UTC m=+2531.412649574"
Oct 30 19:28:59.759100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:28:59.759070    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:29:00.655555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:00.655519    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:29:00.656407 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:00.656389    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:29:01.417367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.417321    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"]
Oct 30 19:29:01.421524 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.421503    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.474591 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.474559    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-c16ba7-serving-cert\""
Oct 30 19:29:01.523980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.523949    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"]
Oct 30 19:29:01.551409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.551370    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlxw2\" (UniqueName: \"kubernetes.io/projected/22d77107-0992-41b9-adfc-2e1f0826a477-kube-api-access-wlxw2\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.551613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.551482    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-metrics-certs\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.551613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.551517    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d77107-0992-41b9-adfc-2e1f0826a477-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.551745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.551651    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.657187 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.657145    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d77107-0992-41b9-adfc-2e1f0826a477-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.657740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.657713    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.658001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.657980    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlxw2\" (UniqueName: \"kubernetes.io/projected/22d77107-0992-41b9-adfc-2e1f0826a477-kube-api-access-wlxw2\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.658368 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.658233    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-metrics-certs\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.661650 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:29:01.661614    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-serving-cert: secret "maas-api-e2e-praxis-b-c16ba7-serving-cert" not found
Oct 30 19:29:01.661776 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:29:01.661699    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls podName:22d77107-0992-41b9-adfc-2e1f0826a477 nodeName:}" failed. No retries permitted until 2026-10-30 19:29:02.16167456 +0000 UTC m=+2534.746157554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls") pod "maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj" (UID: "22d77107-0992-41b9-adfc-2e1f0826a477") : secret "maas-api-e2e-praxis-b-c16ba7-serving-cert" not found
Oct 30 19:29:01.661902 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.661877    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d77107-0992-41b9-adfc-2e1f0826a477-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.666372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.666348    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-metrics-certs\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:01.730937 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:01.730858    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlxw2\" (UniqueName: \"kubernetes.io/projected/22d77107-0992-41b9-adfc-2e1f0826a477-kube-api-access-wlxw2\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:02.166043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:02.166007    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:02.168496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:02.168467    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls\") pod \"maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:02.335553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:02.335512    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:02.713937 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:02.713904    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"]
Oct 30 19:29:02.717506 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:29:02.717466    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d77107_0992_41b9_adfc_2e1f0826a477.slice/crio-14e10a5131f9a8a5227a0dbfb7d5e63c8bd255d5704c75213bf1ac080324bc87 WatchSource:0}: Error finding container 14e10a5131f9a8a5227a0dbfb7d5e63c8bd255d5704c75213bf1ac080324bc87: Status 404 returned error can't find the container with id 14e10a5131f9a8a5227a0dbfb7d5e63c8bd255d5704c75213bf1ac080324bc87
Oct 30 19:29:02.719211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:02.719190    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:29:03.683784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:03.683745    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj" event={"ID":"22d77107-0992-41b9-adfc-2e1f0826a477","Type":"ContainerStarted","Data":"725f64d34302c2daff1eb50768d0d4255f77af8ecc4c43ffa49f77199156c413"}
Oct 30 19:29:03.683985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:03.683806    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj" event={"ID":"22d77107-0992-41b9-adfc-2e1f0826a477","Type":"ContainerStarted","Data":"14e10a5131f9a8a5227a0dbfb7d5e63c8bd255d5704c75213bf1ac080324bc87"}
Oct 30 19:29:03.684068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:03.683979    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:03.827178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:03.827073    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj" podStartSLOduration=2.007848808 podStartE2EDuration="2.827056729s" totalImagesPullingTime="819.207921ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:03.824367367 +0000 UTC m=+2536.408850364" watchObservedRunningTime="2026-10-30 19:29:03.827056729 +0000 UTC m=+2536.411539730"
Oct 30 19:29:09.698290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:09.698259    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:29:16.187044 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:29:16.186998    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:29:27.746969 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.746936    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"]
Oct 30 19:29:27.751761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.751740    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.802385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.802352    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-f9a20d-serving-cert\""
Oct 30 19:29:27.853042 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.853011    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"]
Oct 30 19:29:27.895620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.895581    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-metrics-certs\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.895862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.895675    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.895862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.895719    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwzmv\" (UniqueName: \"kubernetes.io/projected/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-kube-api-access-nwzmv\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.896003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.895877    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.998585 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.998497    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.998769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.998690    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-metrics-certs\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.998769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.998761    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.998936 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.998796    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwzmv\" (UniqueName: \"kubernetes.io/projected/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-kube-api-access-nwzmv\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:27.999288 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:27.999264    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:28.001413 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.001379    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:28.001534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.001472    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-metrics-certs\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:28.057607 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.057579    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwzmv\" (UniqueName: \"kubernetes.io/projected/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-kube-api-access-nwzmv\") pod \"maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:28.066416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.066386    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:28.446561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.446524    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"]
Oct 30 19:29:28.451295 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:29:28.451268    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc40b3f2_ce90_4b44_ba94_6119d3b2e122.slice/crio-d6e744ed3a3faf5b5f0d6a99de3ac68a9bb9516345c395266ab0a01799cb3797 WatchSource:0}: Error finding container d6e744ed3a3faf5b5f0d6a99de3ac68a9bb9516345c395266ab0a01799cb3797: Status 404 returned error can't find the container with id d6e744ed3a3faf5b5f0d6a99de3ac68a9bb9516345c395266ab0a01799cb3797
Oct 30 19:29:28.910513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.910471    5232 generic.go:356] "Generic (PLEG): container finished" podID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerID="90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb" exitCode=0
Oct 30 19:29:28.911005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.910547    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" event={"ID":"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c","Type":"ContainerDied","Data":"90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb"}
Oct 30 19:29:28.911073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.911046    5232 scope.go:122] "RemoveContainer" containerID="90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb"
Oct 30 19:29:28.912024 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.912004    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh" event={"ID":"bc40b3f2-ce90-4b44-ba94-6119d3b2e122","Type":"ContainerStarted","Data":"d6e744ed3a3faf5b5f0d6a99de3ac68a9bb9516345c395266ab0a01799cb3797"}
Oct 30 19:29:28.912470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.912441    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:28.912535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.912514    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:28.912569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:28.912551    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:29.920344 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:29.920305    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" event={"ID":"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c","Type":"ContainerStarted","Data":"e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee"}
Oct 30 19:29:29.922045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:29.922014    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh" event={"ID":"bc40b3f2-ce90-4b44-ba94-6119d3b2e122","Type":"ContainerStarted","Data":"2702dedea0fa0811530f6a670b78ca58df8928a7a4c1d38c6ef1e6b9eae7459b"}
Oct 30 19:29:29.922173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:29.922089    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:30.174363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:30.174265    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh" podStartSLOduration=2.373608403 podStartE2EDuration="3.174249628s" totalImagesPullingTime="800.641225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:30.17301176 +0000 UTC m=+2562.757494770" watchObservedRunningTime="2026-10-30 19:29:30.174249628 +0000 UTC m=+2562.758732625"
Oct 30 19:29:30.368978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:30.368945    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:29:30.369155 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:30.369135    5232 patch_prober.go:29] interesting pod/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:29:30.369198 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:30.369185    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:29:31.369413 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:31.369373    5232 patch_prober.go:29] interesting pod/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:29:31.369913 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:31.369441    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:29:32.369027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:32.368980    5232 patch_prober.go:29] interesting pod/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:29:32.369208 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:32.369051    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:29:33.369405 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:33.369364    5232 patch_prober.go:29] interesting pod/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:29:33.369858 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:33.369428    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:29:34.368862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:34.368786    5232 patch_prober.go:29] interesting pod/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:29:34.369058 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:34.368887    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:29:35.282415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:35.282380    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:29:35.373051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:35.373019    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:29:35.935156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:35.935124    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:29:35.993908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:35.993873    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:29:46.189345 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:29:46.189303    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:29:54.130159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:54.130125    5232 generic.go:356] "Generic (PLEG): container finished" podID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerID="ededd401189e94debc6ed3d260571c9ecd4706a434c96977ad4d169b8815f2aa" exitCode=0
Oct 30 19:29:54.130622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:54.130199    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" event={"ID":"fc314899-ffa0-49eb-a97b-1bde15e33c58","Type":"ContainerDied","Data":"ededd401189e94debc6ed3d260571c9ecd4706a434c96977ad4d169b8815f2aa"}
Oct 30 19:29:54.130622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:54.130578    5232 scope.go:122] "RemoveContainer" containerID="ededd401189e94debc6ed3d260571c9ecd4706a434c96977ad4d169b8815f2aa"
Oct 30 19:29:54.132065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:54.132037    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:54.132129 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:54.132104    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:54.132165 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:54.132148    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:29:55.161838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:55.161567    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" event={"ID":"fc314899-ffa0-49eb-a97b-1bde15e33c58","Type":"ContainerStarted","Data":"f3180eb5c1cbec718626980cd6ca5959b4e3a997567b9d714d099bb44da3d1ac"}
Oct 30 19:29:55.958837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:55.949889    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:29:55.958837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:55.956956    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:29:55.958837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:55.957047    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:29:56.962275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:56.962233    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:29:56.962790 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:56.962303    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:29:57.956837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:57.956187    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:29:57.956837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:57.956250    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:29:58.449140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:58.449105    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:29:58.962221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:58.962178    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:29:58.962512 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:58.962485    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:29:59.959305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:59.959268    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:29:59.959935 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:29:59.959903    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:30:00.192835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.191195    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"]
Oct 30 19:30:00.195669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.195638    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:00.222397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.222320    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"]
Oct 30 19:30:00.227245 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.227223    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:00.246948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.246920    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"]
Oct 30 19:30:00.250354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.250332    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:00.252538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.252514    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"]
Oct 30 19:30:00.256269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.256252    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:00.269962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.269941    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"]
Oct 30 19:30:00.273457 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.273440    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:00.292672 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.292642    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"]
Oct 30 19:30:00.330545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.330470    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 19:30:00.347562 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347521    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-jbzt6\" (UID: \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:00.347749 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347617    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnflc\" (UniqueName: \"kubernetes.io/projected/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-kube-api-access-tnflc\") pod \"maas-api-key-cleanup-29889810-jbzt6\" (UID: \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:00.347749 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347677    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70922e73-0a54-4c22-b079-0a3b7538a22e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb\" (UID: \"70922e73-0a54-4c22-b079-0a3b7538a22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:00.347925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347784    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lgss\" (UniqueName: \"kubernetes.io/projected/66939b12-acdb-4eb8-b426-ee87b793cf28-kube-api-access-8lgss\") pod \"maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc\" (UID: \"66939b12-acdb-4eb8-b426-ee87b793cf28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:00.347925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347830    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47zgh\" (UniqueName: \"kubernetes.io/projected/174636b6-19c0-44b9-8dd5-30a36ce8573f-kube-api-access-47zgh\") pod \"maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh\" (UID: \"174636b6-19c0-44b9-8dd5-30a36ce8573f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:00.347925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347870    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9224ff9-b932-4c67-919c-69def5c59e67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj\" (UID: \"a9224ff9-b932-4c67-919c-69def5c59e67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:00.347925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347901    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r9vk\" (UniqueName: \"kubernetes.io/projected/a9224ff9-b932-4c67-919c-69def5c59e67-kube-api-access-4r9vk\") pod \"maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj\" (UID: \"a9224ff9-b932-4c67-919c-69def5c59e67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:00.348153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347933    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/174636b6-19c0-44b9-8dd5-30a36ce8573f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh\" (UID: \"174636b6-19c0-44b9-8dd5-30a36ce8573f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:00.348153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.347984    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79p5z\" (UniqueName: \"kubernetes.io/projected/70922e73-0a54-4c22-b079-0a3b7538a22e-kube-api-access-79p5z\") pod \"maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb\" (UID: \"70922e73-0a54-4c22-b079-0a3b7538a22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:00.348153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.348060    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66939b12-acdb-4eb8-b426-ee87b793cf28-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc\" (UID: \"66939b12-acdb-4eb8-b426-ee87b793cf28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:00.352899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.352870    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"]
Oct 30 19:30:00.356229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.356199    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"]
Oct 30 19:30:00.359241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.359222    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"]
Oct 30 19:30:00.361605 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.361582    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"]
Oct 30 19:30:00.449283 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449251    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70922e73-0a54-4c22-b079-0a3b7538a22e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb\" (UID: \"70922e73-0a54-4c22-b079-0a3b7538a22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:00.449474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449337    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lgss\" (UniqueName: \"kubernetes.io/projected/66939b12-acdb-4eb8-b426-ee87b793cf28-kube-api-access-8lgss\") pod \"maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc\" (UID: \"66939b12-acdb-4eb8-b426-ee87b793cf28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:00.449474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449359    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47zgh\" (UniqueName: \"kubernetes.io/projected/174636b6-19c0-44b9-8dd5-30a36ce8573f-kube-api-access-47zgh\") pod \"maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh\" (UID: \"174636b6-19c0-44b9-8dd5-30a36ce8573f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:00.449474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449379    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9224ff9-b932-4c67-919c-69def5c59e67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj\" (UID: \"a9224ff9-b932-4c67-919c-69def5c59e67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:00.449474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449398    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r9vk\" (UniqueName: \"kubernetes.io/projected/a9224ff9-b932-4c67-919c-69def5c59e67-kube-api-access-4r9vk\") pod \"maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj\" (UID: \"a9224ff9-b932-4c67-919c-69def5c59e67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:00.449474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449422    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/174636b6-19c0-44b9-8dd5-30a36ce8573f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh\" (UID: \"174636b6-19c0-44b9-8dd5-30a36ce8573f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:00.449474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449466    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79p5z\" (UniqueName: \"kubernetes.io/projected/70922e73-0a54-4c22-b079-0a3b7538a22e-kube-api-access-79p5z\") pod \"maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb\" (UID: \"70922e73-0a54-4c22-b079-0a3b7538a22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:00.450280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449534    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66939b12-acdb-4eb8-b426-ee87b793cf28-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc\" (UID: \"66939b12-acdb-4eb8-b426-ee87b793cf28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:00.450280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449579    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-jbzt6\" (UID: \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:00.450280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.449655    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnflc\" (UniqueName: \"kubernetes.io/projected/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-kube-api-access-tnflc\") pod \"maas-api-key-cleanup-29889810-jbzt6\" (UID: \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:00.450280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.450051    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70922e73-0a54-4c22-b079-0a3b7538a22e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb\" (UID: \"70922e73-0a54-4c22-b079-0a3b7538a22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:00.450470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.450303    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66939b12-acdb-4eb8-b426-ee87b793cf28-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc\" (UID: \"66939b12-acdb-4eb8-b426-ee87b793cf28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:00.450470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.450349    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9224ff9-b932-4c67-919c-69def5c59e67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj\" (UID: \"a9224ff9-b932-4c67-919c-69def5c59e67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:00.450470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.450362    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/174636b6-19c0-44b9-8dd5-30a36ce8573f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh\" (UID: \"174636b6-19c0-44b9-8dd5-30a36ce8573f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:00.450470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.450371    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-jbzt6\" (UID: \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:00.509707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.509638    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r9vk\" (UniqueName: \"kubernetes.io/projected/a9224ff9-b932-4c67-919c-69def5c59e67-kube-api-access-4r9vk\") pod \"maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj\" (UID: \"a9224ff9-b932-4c67-919c-69def5c59e67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:00.509887 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.509702    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnflc\" (UniqueName: \"kubernetes.io/projected/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-kube-api-access-tnflc\") pod \"maas-api-key-cleanup-29889810-jbzt6\" (UID: \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:00.509887 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.509761    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lgss\" (UniqueName: \"kubernetes.io/projected/66939b12-acdb-4eb8-b426-ee87b793cf28-kube-api-access-8lgss\") pod \"maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc\" (UID: \"66939b12-acdb-4eb8-b426-ee87b793cf28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:00.510000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.509978    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47zgh\" (UniqueName: \"kubernetes.io/projected/174636b6-19c0-44b9-8dd5-30a36ce8573f-kube-api-access-47zgh\") pod \"maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh\" (UID: \"174636b6-19c0-44b9-8dd5-30a36ce8573f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:00.510271 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.510255    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79p5z\" (UniqueName: \"kubernetes.io/projected/70922e73-0a54-4c22-b079-0a3b7538a22e-kube-api-access-79p5z\") pod \"maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb\" (UID: \"70922e73-0a54-4c22-b079-0a3b7538a22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:00.514374 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.514360    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:00.542033 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.542011    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:00.565431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.565389    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:00.572059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.572034    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:00.588244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.588212    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:00.895116 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.895089    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"]
Oct 30 19:30:00.896945 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:30:00.896917    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66939b12_acdb_4eb8_b426_ee87b793cf28.slice/crio-8b1592be6af3a585e6908fb3703df9d5a608584f4cf747f6eef7e96e61a92beb WatchSource:0}: Error finding container 8b1592be6af3a585e6908fb3703df9d5a608584f4cf747f6eef7e96e61a92beb: Status 404 returned error can't find the container with id 8b1592be6af3a585e6908fb3703df9d5a608584f4cf747f6eef7e96e61a92beb
Oct 30 19:30:00.946281 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.946257    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"]
Oct 30 19:30:00.955627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.955602    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:30:00.956949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:00.956925    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:30:00.959096 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:30:00.959067    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9224ff9_b932_4c67_919c_69def5c59e67.slice/crio-c877b758d55068b6fda410e27b6b4e6809d469ebedcf4b0b70e561f0831a4d1d WatchSource:0}: Error finding container c877b758d55068b6fda410e27b6b4e6809d469ebedcf4b0b70e561f0831a4d1d: Status 404 returned error can't find the container with id c877b758d55068b6fda410e27b6b4e6809d469ebedcf4b0b70e561f0831a4d1d
Oct 30 19:30:01.000292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:01.000261    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"]
Oct 30 19:30:01.021268 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:01.021237    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"]
Oct 30 19:30:01.028637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:01.028615    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"]
Oct 30 19:30:01.233265 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:01.233156    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" event={"ID":"a9224ff9-b932-4c67-919c-69def5c59e67","Type":"ContainerStarted","Data":"c877b758d55068b6fda410e27b6b4e6809d469ebedcf4b0b70e561f0831a4d1d"}
Oct 30 19:30:01.235078 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:01.235052    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" event={"ID":"66939b12-acdb-4eb8-b426-ee87b793cf28","Type":"ContainerStarted","Data":"8b1592be6af3a585e6908fb3703df9d5a608584f4cf747f6eef7e96e61a92beb"}
Oct 30 19:30:01.236926 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:01.236899    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" event={"ID":"174636b6-19c0-44b9-8dd5-30a36ce8573f","Type":"ContainerStarted","Data":"00c2131b925f1560603248b6693b8b80f2ba79d21de9739bece493661cff510e"}
Oct 30 19:30:01.240604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:01.240577    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" event={"ID":"70922e73-0a54-4c22-b079-0a3b7538a22e","Type":"ContainerStarted","Data":"60a5a6686a8508897a2cf5b75a87a99a3f70e00600c03ef1156f5545efb1f588"}
Oct 30 19:30:01.242723 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:01.242699    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" event={"ID":"eae7d006-10ba-4f0c-91fb-f4bfe6a97419","Type":"ContainerStarted","Data":"d1c4011d208d86b8fde6fb636b5ac2905abd0c4a990f44f55be1f9e6b9a4b2d7"}
Oct 30 19:30:02.254457 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.254415    5232 generic.go:356] "Generic (PLEG): container finished" podID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerID="60dafa4fb3658b0f11b16c8fb9a5e43c328ed9bf3889020c8a874187eb9cb16e" exitCode=60
Oct 30 19:30:02.254910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.254503    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" event={"ID":"eae7d006-10ba-4f0c-91fb-f4bfe6a97419","Type":"ContainerDied","Data":"60dafa4fb3658b0f11b16c8fb9a5e43c328ed9bf3889020c8a874187eb9cb16e"}
Oct 30 19:30:02.254910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.254768    5232 scope.go:122] "RemoveContainer" containerID="60dafa4fb3658b0f11b16c8fb9a5e43c328ed9bf3889020c8a874187eb9cb16e"
Oct 30 19:30:02.256604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.256457    5232 generic.go:356] "Generic (PLEG): container finished" podID="a9224ff9-b932-4c67-919c-69def5c59e67" containerID="3fd366d7808a91d98a86d36c5f941e53052802d75c7009877102a5756bbf129c" exitCode=60
Oct 30 19:30:02.256604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.256532    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" event={"ID":"a9224ff9-b932-4c67-919c-69def5c59e67","Type":"ContainerDied","Data":"3fd366d7808a91d98a86d36c5f941e53052802d75c7009877102a5756bbf129c"}
Oct 30 19:30:02.256773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.256756    5232 scope.go:122] "RemoveContainer" containerID="3fd366d7808a91d98a86d36c5f941e53052802d75c7009877102a5756bbf129c"
Oct 30 19:30:02.258181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.258166    5232 generic.go:356] "Generic (PLEG): container finished" podID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerID="35e269169dbe368fa98daf81331fc164b9c43e885c2a2fae10748cc54a285177" exitCode=60
Oct 30 19:30:02.258282 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.258249    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" event={"ID":"66939b12-acdb-4eb8-b426-ee87b793cf28","Type":"ContainerDied","Data":"35e269169dbe368fa98daf81331fc164b9c43e885c2a2fae10748cc54a285177"}
Oct 30 19:30:02.258476 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.258404    5232 scope.go:122] "RemoveContainer" containerID="35e269169dbe368fa98daf81331fc164b9c43e885c2a2fae10748cc54a285177"
Oct 30 19:30:02.260223 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.260203    5232 generic.go:356] "Generic (PLEG): container finished" podID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerID="7e17ae19e4aa02ec0443d3e88379c457e2bcbfc82c7ac8b50ad686d8f1156262" exitCode=60
Oct 30 19:30:02.260312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.260268    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" event={"ID":"174636b6-19c0-44b9-8dd5-30a36ce8573f","Type":"ContainerDied","Data":"7e17ae19e4aa02ec0443d3e88379c457e2bcbfc82c7ac8b50ad686d8f1156262"}
Oct 30 19:30:02.260444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.260431    5232 scope.go:122] "RemoveContainer" containerID="7e17ae19e4aa02ec0443d3e88379c457e2bcbfc82c7ac8b50ad686d8f1156262"
Oct 30 19:30:02.261860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.261842    5232 generic.go:356] "Generic (PLEG): container finished" podID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerID="f75110ecb626ca7eb1e9bbe4039a458d6437b46970855154ea5a26112ce618c7" exitCode=60
Oct 30 19:30:02.261978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.261877    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" event={"ID":"70922e73-0a54-4c22-b079-0a3b7538a22e","Type":"ContainerDied","Data":"f75110ecb626ca7eb1e9bbe4039a458d6437b46970855154ea5a26112ce618c7"}
Oct 30 19:30:02.262105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:02.262089    5232 scope.go:122] "RemoveContainer" containerID="f75110ecb626ca7eb1e9bbe4039a458d6437b46970855154ea5a26112ce618c7"
Oct 30 19:30:03.270617 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.270585    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" event={"ID":"eae7d006-10ba-4f0c-91fb-f4bfe6a97419","Type":"ContainerStarted","Data":"97eb35bafac9f3d39bf3635c520587129d3a84e41324af0571a9cbaa731c1fca"}
Oct 30 19:30:03.272536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.272509    5232 generic.go:356] "Generic (PLEG): container finished" podID="a9224ff9-b932-4c67-919c-69def5c59e67" containerID="b26e24c26b9057769c340d3b3f5cdf55de95f650dfd6d43589e3dd6ee41bda4e" exitCode=60
Oct 30 19:30:03.272641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.272596    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" event={"ID":"a9224ff9-b932-4c67-919c-69def5c59e67","Type":"ContainerDied","Data":"b26e24c26b9057769c340d3b3f5cdf55de95f650dfd6d43589e3dd6ee41bda4e"}
Oct 30 19:30:03.272707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.272651    5232 scope.go:122] "RemoveContainer" containerID="3fd366d7808a91d98a86d36c5f941e53052802d75c7009877102a5756bbf129c"
Oct 30 19:30:03.272938 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.272917    5232 scope.go:122] "RemoveContainer" containerID="b26e24c26b9057769c340d3b3f5cdf55de95f650dfd6d43589e3dd6ee41bda4e"
Oct 30 19:30:03.273247 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:03.273221    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj_odh-ai-gateway-infra(a9224ff9-b932-4c67-919c-69def5c59e67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" podUID="a9224ff9-b932-4c67-919c-69def5c59e67"
Oct 30 19:30:03.274797 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.274773    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" event={"ID":"66939b12-acdb-4eb8-b426-ee87b793cf28","Type":"ContainerStarted","Data":"3bc0231a0f0765e4db6a43eb8882afe53b789f652460c1f9e136bd6aac67d05f"}
Oct 30 19:30:03.276554 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.276530    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" event={"ID":"174636b6-19c0-44b9-8dd5-30a36ce8573f","Type":"ContainerStarted","Data":"b520717cb2022c2c67539647c38575d2652a25c474c85408f3d02f2e97da9f56"}
Oct 30 19:30:03.278384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.278365    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" event={"ID":"70922e73-0a54-4c22-b079-0a3b7538a22e","Type":"ContainerStarted","Data":"7419859e5a0ecfd6810b0950c7f91903130b83a3489814bebf04d1c380bfc986"}
Oct 30 19:30:03.383214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.383153    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" podStartSLOduration=3.383135737 podStartE2EDuration="3.383135737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.381734734 +0000 UTC m=+2595.966217733" watchObservedRunningTime="2026-10-30 19:30:03.383135737 +0000 UTC m=+2595.967618734"
Oct 30 19:30:03.608982 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.608928    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" podStartSLOduration=3.608914257 podStartE2EDuration="3.608914257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.606475042 +0000 UTC m=+2596.190958038" watchObservedRunningTime="2026-10-30 19:30:03.608914257 +0000 UTC m=+2596.193397254"
Oct 30 19:30:03.723061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.722941    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" podStartSLOduration=3.722924231 podStartE2EDuration="3.722924231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.718758885 +0000 UTC m=+2596.303241896" watchObservedRunningTime="2026-10-30 19:30:03.722924231 +0000 UTC m=+2596.307407231"
Oct 30 19:30:03.831546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:03.831491    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" podStartSLOduration=3.831477339 podStartE2EDuration="3.831477339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.829835243 +0000 UTC m=+2596.414318236" watchObservedRunningTime="2026-10-30 19:30:03.831477339 +0000 UTC m=+2596.415960336"
Oct 30 19:30:04.287497 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.287422    5232 scope.go:122] "RemoveContainer" containerID="b26e24c26b9057769c340d3b3f5cdf55de95f650dfd6d43589e3dd6ee41bda4e"
Oct 30 19:30:04.288017 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:04.287771    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj_odh-ai-gateway-infra(a9224ff9-b932-4c67-919c-69def5c59e67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" podUID="a9224ff9-b932-4c67-919c-69def5c59e67"
Oct 30 19:30:04.289116 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.289090    5232 generic.go:356] "Generic (PLEG): container finished" podID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerID="3bc0231a0f0765e4db6a43eb8882afe53b789f652460c1f9e136bd6aac67d05f" exitCode=60
Oct 30 19:30:04.289226 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.289167    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" event={"ID":"66939b12-acdb-4eb8-b426-ee87b793cf28","Type":"ContainerDied","Data":"3bc0231a0f0765e4db6a43eb8882afe53b789f652460c1f9e136bd6aac67d05f"}
Oct 30 19:30:04.289309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.289248    5232 scope.go:122] "RemoveContainer" containerID="35e269169dbe368fa98daf81331fc164b9c43e885c2a2fae10748cc54a285177"
Oct 30 19:30:04.289403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.289388    5232 scope.go:122] "RemoveContainer" containerID="3bc0231a0f0765e4db6a43eb8882afe53b789f652460c1f9e136bd6aac67d05f"
Oct 30 19:30:04.289674 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:04.289649    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc_odh-ai-gateway-infra(66939b12-acdb-4eb8-b426-ee87b793cf28)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28"
Oct 30 19:30:04.291122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.291102    5232 generic.go:356] "Generic (PLEG): container finished" podID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerID="b520717cb2022c2c67539647c38575d2652a25c474c85408f3d02f2e97da9f56" exitCode=60
Oct 30 19:30:04.291203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.291183    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" event={"ID":"174636b6-19c0-44b9-8dd5-30a36ce8573f","Type":"ContainerDied","Data":"b520717cb2022c2c67539647c38575d2652a25c474c85408f3d02f2e97da9f56"}
Oct 30 19:30:04.291393 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.291378    5232 scope.go:122] "RemoveContainer" containerID="b520717cb2022c2c67539647c38575d2652a25c474c85408f3d02f2e97da9f56"
Oct 30 19:30:04.291667 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:04.291644    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh_odh-ai-gateway-infra(174636b6-19c0-44b9-8dd5-30a36ce8573f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f"
Oct 30 19:30:04.293114 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.293091    5232 generic.go:356] "Generic (PLEG): container finished" podID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerID="7419859e5a0ecfd6810b0950c7f91903130b83a3489814bebf04d1c380bfc986" exitCode=60
Oct 30 19:30:04.293196 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.293174    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" event={"ID":"70922e73-0a54-4c22-b079-0a3b7538a22e","Type":"ContainerDied","Data":"7419859e5a0ecfd6810b0950c7f91903130b83a3489814bebf04d1c380bfc986"}
Oct 30 19:30:04.293435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.293416    5232 scope.go:122] "RemoveContainer" containerID="7419859e5a0ecfd6810b0950c7f91903130b83a3489814bebf04d1c380bfc986"
Oct 30 19:30:04.293737 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:04.293712    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb_odh-ai-gateway-infra(70922e73-0a54-4c22-b079-0a3b7538a22e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e"
Oct 30 19:30:04.294992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.294975    5232 generic.go:356] "Generic (PLEG): container finished" podID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerID="97eb35bafac9f3d39bf3635c520587129d3a84e41324af0571a9cbaa731c1fca" exitCode=60
Oct 30 19:30:04.295077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.295040    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" event={"ID":"eae7d006-10ba-4f0c-91fb-f4bfe6a97419","Type":"ContainerDied","Data":"97eb35bafac9f3d39bf3635c520587129d3a84e41324af0571a9cbaa731c1fca"}
Oct 30 19:30:04.295265 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.295241    5232 scope.go:122] "RemoveContainer" containerID="97eb35bafac9f3d39bf3635c520587129d3a84e41324af0571a9cbaa731c1fca"
Oct 30 19:30:04.295462 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:04.295445    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-jbzt6_odh-ai-gateway-infra(eae7d006-10ba-4f0c-91fb-f4bfe6a97419)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419"
Oct 30 19:30:04.301973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.301950    5232 scope.go:122] "RemoveContainer" containerID="7e17ae19e4aa02ec0443d3e88379c457e2bcbfc82c7ac8b50ad686d8f1156262"
Oct 30 19:30:04.313454 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.313431    5232 scope.go:122] "RemoveContainer" containerID="f75110ecb626ca7eb1e9bbe4039a458d6437b46970855154ea5a26112ce618c7"
Oct 30 19:30:04.325789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:04.325760    5232 scope.go:122] "RemoveContainer" containerID="60dafa4fb3658b0f11b16c8fb9a5e43c328ed9bf3889020c8a874187eb9cb16e"
Oct 30 19:30:05.303673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:05.303642    5232 scope.go:122] "RemoveContainer" containerID="3bc0231a0f0765e4db6a43eb8882afe53b789f652460c1f9e136bd6aac67d05f"
Oct 30 19:30:05.304235 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:05.304067    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc_odh-ai-gateway-infra(66939b12-acdb-4eb8-b426-ee87b793cf28)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28"
Oct 30 19:30:05.305138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:05.305114    5232 scope.go:122] "RemoveContainer" containerID="b520717cb2022c2c67539647c38575d2652a25c474c85408f3d02f2e97da9f56"
Oct 30 19:30:05.305408 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:05.305381    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh_odh-ai-gateway-infra(174636b6-19c0-44b9-8dd5-30a36ce8573f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f"
Oct 30 19:30:05.306563 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:05.306545    5232 scope.go:122] "RemoveContainer" containerID="7419859e5a0ecfd6810b0950c7f91903130b83a3489814bebf04d1c380bfc986"
Oct 30 19:30:05.306796 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:05.306778    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb_odh-ai-gateway-infra(70922e73-0a54-4c22-b079-0a3b7538a22e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e"
Oct 30 19:30:05.308059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:05.308044    5232 scope.go:122] "RemoveContainer" containerID="97eb35bafac9f3d39bf3635c520587129d3a84e41324af0571a9cbaa731c1fca"
Oct 30 19:30:05.308238 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:05.308223    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-jbzt6_odh-ai-gateway-infra(eae7d006-10ba-4f0c-91fb-f4bfe6a97419)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419"
Oct 30 19:30:13.533571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:13.533492    5232 scope.go:122] "RemoveContainer" containerID="b26e24c26b9057769c340d3b3f5cdf55de95f650dfd6d43589e3dd6ee41bda4e"
Oct 30 19:30:14.402291 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:14.402255    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" event={"ID":"a9224ff9-b932-4c67-919c-69def5c59e67","Type":"ContainerStarted","Data":"4337f705a6cc9d54db957991abe65929bd5b97d5b504ff4d05956f77ff9264c7"}
Oct 30 19:30:14.532960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:14.532931    5232 scope.go:122] "RemoveContainer" containerID="97eb35bafac9f3d39bf3635c520587129d3a84e41324af0571a9cbaa731c1fca"
Oct 30 19:30:15.411912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:15.411874    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" event={"ID":"eae7d006-10ba-4f0c-91fb-f4bfe6a97419","Type":"ContainerStarted","Data":"15a8b29f1372deb227cac3c1e3ea705d2bfd8f45f96f31eb1ff0833da48c7773"}
Oct 30 19:30:15.413995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:15.413969    5232 generic.go:356] "Generic (PLEG): container finished" podID="a9224ff9-b932-4c67-919c-69def5c59e67" containerID="4337f705a6cc9d54db957991abe65929bd5b97d5b504ff4d05956f77ff9264c7" exitCode=60
Oct 30 19:30:15.414118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:15.414036    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" event={"ID":"a9224ff9-b932-4c67-919c-69def5c59e67","Type":"ContainerDied","Data":"4337f705a6cc9d54db957991abe65929bd5b97d5b504ff4d05956f77ff9264c7"}
Oct 30 19:30:15.414118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:15.414090    5232 scope.go:122] "RemoveContainer" containerID="b26e24c26b9057769c340d3b3f5cdf55de95f650dfd6d43589e3dd6ee41bda4e"
Oct 30 19:30:15.414364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:15.414352    5232 scope.go:122] "RemoveContainer" containerID="4337f705a6cc9d54db957991abe65929bd5b97d5b504ff4d05956f77ff9264c7"
Oct 30 19:30:15.414617 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:15.414599    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj_odh-ai-gateway-infra(a9224ff9-b932-4c67-919c-69def5c59e67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" podUID="a9224ff9-b932-4c67-919c-69def5c59e67"
Oct 30 19:30:15.526481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:15.526429    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" podStartSLOduration=15.52641429 podStartE2EDuration="15.52641429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:14.564070484 +0000 UTC m=+2607.148553484" watchObservedRunningTime="2026-10-30 19:30:15.52641429 +0000 UTC m=+2608.110897283"
Oct 30 19:30:15.558788 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:15.558753    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"]
Oct 30 19:30:16.188962 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:16.188924    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:30:16.427081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.427046    5232 generic.go:356] "Generic (PLEG): container finished" podID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerID="15a8b29f1372deb227cac3c1e3ea705d2bfd8f45f96f31eb1ff0833da48c7773" exitCode=60
Oct 30 19:30:16.427539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.427092    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" event={"ID":"eae7d006-10ba-4f0c-91fb-f4bfe6a97419","Type":"ContainerDied","Data":"15a8b29f1372deb227cac3c1e3ea705d2bfd8f45f96f31eb1ff0833da48c7773"}
Oct 30 19:30:16.427539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.427164    5232 scope.go:122] "RemoveContainer" containerID="97eb35bafac9f3d39bf3635c520587129d3a84e41324af0571a9cbaa731c1fca"
Oct 30 19:30:16.427998 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.427969    5232 scope.go:122] "RemoveContainer" containerID="15a8b29f1372deb227cac3c1e3ea705d2bfd8f45f96f31eb1ff0833da48c7773"
Oct 30 19:30:16.428379 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:16.428352    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-jbzt6_odh-ai-gateway-infra(eae7d006-10ba-4f0c-91fb-f4bfe6a97419)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419"
Oct 30 19:30:16.533494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.533112    5232 scope.go:122] "RemoveContainer" containerID="7419859e5a0ecfd6810b0950c7f91903130b83a3489814bebf04d1c380bfc986"
Oct 30 19:30:16.534961 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.534023    5232 scope.go:122] "RemoveContainer" containerID="b520717cb2022c2c67539647c38575d2652a25c474c85408f3d02f2e97da9f56"
Oct 30 19:30:16.538652 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.538626    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"]
Oct 30 19:30:16.601950 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.601918    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:30:16.602103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.601970    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:30:16.639747 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.639723    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:16.799667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.799633    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9224ff9-b932-4c67-919c-69def5c59e67-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9224ff9-b932-4c67-919c-69def5c59e67-maas-api-service-ca\") pod \"a9224ff9-b932-4c67-919c-69def5c59e67\" (UID: \"a9224ff9-b932-4c67-919c-69def5c59e67\") "
Oct 30 19:30:16.799880 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.799705    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9224ff9-b932-4c67-919c-69def5c59e67-kube-api-access-4r9vk\" (UniqueName: \"kubernetes.io/projected/a9224ff9-b932-4c67-919c-69def5c59e67-kube-api-access-4r9vk\") pod \"a9224ff9-b932-4c67-919c-69def5c59e67\" (UID: \"a9224ff9-b932-4c67-919c-69def5c59e67\") "
Oct 30 19:30:16.800142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.800111    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9224ff9-b932-4c67-919c-69def5c59e67-maas-api-service-ca" pod "a9224ff9-b932-4c67-919c-69def5c59e67" (UID: "a9224ff9-b932-4c67-919c-69def5c59e67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:16.800401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.800374    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9224ff9-b932-4c67-919c-69def5c59e67-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:16.802761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.802738    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9224ff9-b932-4c67-919c-69def5c59e67-kube-api-access-4r9vk" pod "a9224ff9-b932-4c67-919c-69def5c59e67" (UID: "a9224ff9-b932-4c67-919c-69def5c59e67"). InnerVolumeSpecName "kube-api-access-4r9vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:16.901514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:16.901473    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r9vk\" (UniqueName: \"kubernetes.io/projected/a9224ff9-b932-4c67-919c-69def5c59e67-kube-api-access-4r9vk\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:17.435513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.435472    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" event={"ID":"174636b6-19c0-44b9-8dd5-30a36ce8573f","Type":"ContainerStarted","Data":"6e345fe185cb630fdfdd4b2d258b9b5c7f300f5e58e208f427a78f683b05fd42"}
Oct 30 19:30:17.438678 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.437690    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" event={"ID":"70922e73-0a54-4c22-b079-0a3b7538a22e","Type":"ContainerStarted","Data":"ccb9db20b3bf8c6a9dfc5c4c03cb590e118247e8ce899bd6b6fd24d46302824f"}
Oct 30 19:30:17.443113 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.443064    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj" event={"ID":"a9224ff9-b932-4c67-919c-69def5c59e67","Type":"ContainerDied","Data":"c877b758d55068b6fda410e27b6b4e6809d469ebedcf4b0b70e561f0831a4d1d"}
Oct 30 19:30:17.443384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.443364    5232 scope.go:122] "RemoveContainer" containerID="4337f705a6cc9d54db957991abe65929bd5b97d5b504ff4d05956f77ff9264c7"
Oct 30 19:30:17.443624 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.443107    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"
Oct 30 19:30:17.534372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.533371    5232 scope.go:122] "RemoveContainer" containerID="3bc0231a0f0765e4db6a43eb8882afe53b789f652460c1f9e136bd6aac67d05f"
Oct 30 19:30:17.650992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.650960    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:17.813879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.813777    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-maas-api-service-ca\") pod \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\" (UID: \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\") "
Oct 30 19:30:17.814062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.814017    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-kube-api-access-tnflc\" (UniqueName: \"kubernetes.io/projected/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-kube-api-access-tnflc\") pod \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\" (UID: \"eae7d006-10ba-4f0c-91fb-f4bfe6a97419\") "
Oct 30 19:30:17.814210 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.814181    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-maas-api-service-ca" pod "eae7d006-10ba-4f0c-91fb-f4bfe6a97419" (UID: "eae7d006-10ba-4f0c-91fb-f4bfe6a97419"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:17.814465 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.814448    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:17.816202 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.816175    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-kube-api-access-tnflc" pod "eae7d006-10ba-4f0c-91fb-f4bfe6a97419" (UID: "eae7d006-10ba-4f0c-91fb-f4bfe6a97419"). InnerVolumeSpecName "kube-api-access-tnflc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:17.890779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.890740    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"]
Oct 30 19:30:17.895005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.894977    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-c16ba7-29889810-drkfj"]
Oct 30 19:30:17.915447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:17.915404    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnflc\" (UniqueName: \"kubernetes.io/projected/eae7d006-10ba-4f0c-91fb-f4bfe6a97419-kube-api-access-tnflc\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:18.452737 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.452704    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" event={"ID":"66939b12-acdb-4eb8-b426-ee87b793cf28","Type":"ContainerStarted","Data":"2c0c9da2ec78b65f6c95ae80587cbca0408dcc621840fe080ed5154b1ea4e463"}
Oct 30 19:30:18.454698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.454671    5232 generic.go:356] "Generic (PLEG): container finished" podID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerID="6e345fe185cb630fdfdd4b2d258b9b5c7f300f5e58e208f427a78f683b05fd42" exitCode=60
Oct 30 19:30:18.454883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.454740    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" event={"ID":"174636b6-19c0-44b9-8dd5-30a36ce8573f","Type":"ContainerDied","Data":"6e345fe185cb630fdfdd4b2d258b9b5c7f300f5e58e208f427a78f683b05fd42"}
Oct 30 19:30:18.454883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.454794    5232 scope.go:122] "RemoveContainer" containerID="b520717cb2022c2c67539647c38575d2652a25c474c85408f3d02f2e97da9f56"
Oct 30 19:30:18.455184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.455168    5232 scope.go:122] "RemoveContainer" containerID="6e345fe185cb630fdfdd4b2d258b9b5c7f300f5e58e208f427a78f683b05fd42"
Oct 30 19:30:18.455502 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:18.455475    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh_odh-ai-gateway-infra(174636b6-19c0-44b9-8dd5-30a36ce8573f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f"
Oct 30 19:30:18.456777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.456720    5232 generic.go:356] "Generic (PLEG): container finished" podID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerID="ccb9db20b3bf8c6a9dfc5c4c03cb590e118247e8ce899bd6b6fd24d46302824f" exitCode=60
Oct 30 19:30:18.456888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.456789    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" event={"ID":"70922e73-0a54-4c22-b079-0a3b7538a22e","Type":"ContainerDied","Data":"ccb9db20b3bf8c6a9dfc5c4c03cb590e118247e8ce899bd6b6fd24d46302824f"}
Oct 30 19:30:18.457166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.457150    5232 scope.go:122] "RemoveContainer" containerID="ccb9db20b3bf8c6a9dfc5c4c03cb590e118247e8ce899bd6b6fd24d46302824f"
Oct 30 19:30:18.457438 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:18.457414    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb_odh-ai-gateway-infra(70922e73-0a54-4c22-b079-0a3b7538a22e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e"
Oct 30 19:30:18.458340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.458320    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"
Oct 30 19:30:18.458449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.458367    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6" event={"ID":"eae7d006-10ba-4f0c-91fb-f4bfe6a97419","Type":"ContainerDied","Data":"d1c4011d208d86b8fde6fb636b5ac2905abd0c4a990f44f55be1f9e6b9a4b2d7"}
Oct 30 19:30:18.467741 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.467714    5232 scope.go:122] "RemoveContainer" containerID="7419859e5a0ecfd6810b0950c7f91903130b83a3489814bebf04d1c380bfc986"
Oct 30 19:30:18.483036 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.483009    5232 scope.go:122] "RemoveContainer" containerID="15a8b29f1372deb227cac3c1e3ea705d2bfd8f45f96f31eb1ff0833da48c7773"
Oct 30 19:30:18.559611 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.559583    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"]
Oct 30 19:30:18.683366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.683321    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"]
Oct 30 19:30:18.806174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.806140    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"]
Oct 30 19:30:18.814156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:18.814125    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-jbzt6"]
Oct 30 19:30:19.466452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.466414    5232 generic.go:356] "Generic (PLEG): container finished" podID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerID="2c0c9da2ec78b65f6c95ae80587cbca0408dcc621840fe080ed5154b1ea4e463" exitCode=60
Oct 30 19:30:19.466971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.466486    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" event={"ID":"66939b12-acdb-4eb8-b426-ee87b793cf28","Type":"ContainerDied","Data":"2c0c9da2ec78b65f6c95ae80587cbca0408dcc621840fe080ed5154b1ea4e463"}
Oct 30 19:30:19.466971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.466545    5232 scope.go:122] "RemoveContainer" containerID="3bc0231a0f0765e4db6a43eb8882afe53b789f652460c1f9e136bd6aac67d05f"
Oct 30 19:30:19.466971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.466956    5232 scope.go:122] "RemoveContainer" containerID="2c0c9da2ec78b65f6c95ae80587cbca0408dcc621840fe080ed5154b1ea4e463"
Oct 30 19:30:19.467311 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:19.467283    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc_odh-ai-gateway-infra(66939b12-acdb-4eb8-b426-ee87b793cf28)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28"
Oct 30 19:30:19.473180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.473149    5232 generic.go:356] "Generic (PLEG): container finished" podID="73a453db-4937-4dab-8254-c2decafd1fb8" containerID="fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f" exitCode=0
Oct 30 19:30:19.473274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.473179    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" event={"ID":"73a453db-4937-4dab-8254-c2decafd1fb8","Type":"ContainerDied","Data":"fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f"}
Oct 30 19:30:19.473608 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.473593    5232 scope.go:122] "RemoveContainer" containerID="fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f"
Oct 30 19:30:19.475185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.475150    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:19.475275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.475247    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:19.475319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.475286    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:19.549341 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.549311    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" path="/var/lib/kubelet/pods/a9224ff9-b932-4c67-919c-69def5c59e67/volumes"
Oct 30 19:30:19.550002 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.549959    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" path="/var/lib/kubelet/pods/eae7d006-10ba-4f0c-91fb-f4bfe6a97419/volumes"
Oct 30 19:30:19.584832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.584788    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"]
Oct 30 19:30:19.656729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.656702    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:19.696685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.696656    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:19.841423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.841395    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/174636b6-19c0-44b9-8dd5-30a36ce8573f-kube-api-access-47zgh\" (UniqueName: \"kubernetes.io/projected/174636b6-19c0-44b9-8dd5-30a36ce8573f-kube-api-access-47zgh\") pod \"174636b6-19c0-44b9-8dd5-30a36ce8573f\" (UID: \"174636b6-19c0-44b9-8dd5-30a36ce8573f\") "
Oct 30 19:30:19.841600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.841461    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/174636b6-19c0-44b9-8dd5-30a36ce8573f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/174636b6-19c0-44b9-8dd5-30a36ce8573f-maas-api-service-ca\") pod \"174636b6-19c0-44b9-8dd5-30a36ce8573f\" (UID: \"174636b6-19c0-44b9-8dd5-30a36ce8573f\") "
Oct 30 19:30:19.841600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.841536    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70922e73-0a54-4c22-b079-0a3b7538a22e-kube-api-access-79p5z\" (UniqueName: \"kubernetes.io/projected/70922e73-0a54-4c22-b079-0a3b7538a22e-kube-api-access-79p5z\") pod \"70922e73-0a54-4c22-b079-0a3b7538a22e\" (UID: \"70922e73-0a54-4c22-b079-0a3b7538a22e\") "
Oct 30 19:30:19.841765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.841748    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70922e73-0a54-4c22-b079-0a3b7538a22e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70922e73-0a54-4c22-b079-0a3b7538a22e-maas-api-service-ca\") pod \"70922e73-0a54-4c22-b079-0a3b7538a22e\" (UID: \"70922e73-0a54-4c22-b079-0a3b7538a22e\") "
Oct 30 19:30:19.841973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.841935    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/174636b6-19c0-44b9-8dd5-30a36ce8573f-maas-api-service-ca" pod "174636b6-19c0-44b9-8dd5-30a36ce8573f" (UID: "174636b6-19c0-44b9-8dd5-30a36ce8573f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:19.842155 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.842134    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70922e73-0a54-4c22-b079-0a3b7538a22e-maas-api-service-ca" pod "70922e73-0a54-4c22-b079-0a3b7538a22e" (UID: "70922e73-0a54-4c22-b079-0a3b7538a22e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:19.842352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.842335    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70922e73-0a54-4c22-b079-0a3b7538a22e-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:19.842426 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.842354    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/174636b6-19c0-44b9-8dd5-30a36ce8573f-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:19.843744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.843719    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/174636b6-19c0-44b9-8dd5-30a36ce8573f-kube-api-access-47zgh" pod "174636b6-19c0-44b9-8dd5-30a36ce8573f" (UID: "174636b6-19c0-44b9-8dd5-30a36ce8573f"). InnerVolumeSpecName "kube-api-access-47zgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:19.843867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.843756    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70922e73-0a54-4c22-b079-0a3b7538a22e-kube-api-access-79p5z" pod "70922e73-0a54-4c22-b079-0a3b7538a22e" (UID: "70922e73-0a54-4c22-b079-0a3b7538a22e"). InnerVolumeSpecName "kube-api-access-79p5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:19.943669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.943634    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79p5z\" (UniqueName: \"kubernetes.io/projected/70922e73-0a54-4c22-b079-0a3b7538a22e-kube-api-access-79p5z\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:19.943669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:19.943665    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47zgh\" (UniqueName: \"kubernetes.io/projected/174636b6-19c0-44b9-8dd5-30a36ce8573f-kube-api-access-47zgh\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.483100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.483074    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"
Oct 30 19:30:20.483517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.483069    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh" event={"ID":"174636b6-19c0-44b9-8dd5-30a36ce8573f","Type":"ContainerDied","Data":"00c2131b925f1560603248b6693b8b80f2ba79d21de9739bece493661cff510e"}
Oct 30 19:30:20.483517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.483254    5232 scope.go:122] "RemoveContainer" containerID="6e345fe185cb630fdfdd4b2d258b9b5c7f300f5e58e208f427a78f683b05fd42"
Oct 30 19:30:20.484576 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.484544    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"
Oct 30 19:30:20.484671 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.484540    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb" event={"ID":"70922e73-0a54-4c22-b079-0a3b7538a22e","Type":"ContainerDied","Data":"60a5a6686a8508897a2cf5b75a87a99a3f70e00600c03ef1156f5545efb1f588"}
Oct 30 19:30:20.487243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.487211    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" event={"ID":"73a453db-4937-4dab-8254-c2decafd1fb8","Type":"ContainerStarted","Data":"89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150"}
Oct 30 19:30:20.504904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.498975    5232 scope.go:122] "RemoveContainer" containerID="ccb9db20b3bf8c6a9dfc5c4c03cb590e118247e8ce899bd6b6fd24d46302824f"
Oct 30 19:30:20.681647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.681618    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:20.709606 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.709565    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:30:20.709913 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.709890    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:30:20.709970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.709943    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:30:20.721000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.720973    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"]
Oct 30 19:30:20.727659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.727633    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-f9a20d-29889810-b25lh"]
Oct 30 19:30:20.768453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.768358    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66939b12-acdb-4eb8-b426-ee87b793cf28-kube-api-access-8lgss\" (UniqueName: \"kubernetes.io/projected/66939b12-acdb-4eb8-b426-ee87b793cf28-kube-api-access-8lgss\") pod \"66939b12-acdb-4eb8-b426-ee87b793cf28\" (UID: \"66939b12-acdb-4eb8-b426-ee87b793cf28\") "
Oct 30 19:30:20.768453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.768444    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66939b12-acdb-4eb8-b426-ee87b793cf28-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66939b12-acdb-4eb8-b426-ee87b793cf28-maas-api-service-ca\") pod \"66939b12-acdb-4eb8-b426-ee87b793cf28\" (UID: \"66939b12-acdb-4eb8-b426-ee87b793cf28\") "
Oct 30 19:30:20.768770 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.768748    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66939b12-acdb-4eb8-b426-ee87b793cf28-maas-api-service-ca" pod "66939b12-acdb-4eb8-b426-ee87b793cf28" (UID: "66939b12-acdb-4eb8-b426-ee87b793cf28"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:20.768981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.768964    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66939b12-acdb-4eb8-b426-ee87b793cf28-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.770508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.770488    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66939b12-acdb-4eb8-b426-ee87b793cf28-kube-api-access-8lgss" pod "66939b12-acdb-4eb8-b426-ee87b793cf28" (UID: "66939b12-acdb-4eb8-b426-ee87b793cf28"). InnerVolumeSpecName "kube-api-access-8lgss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:20.870402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.870364    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lgss\" (UniqueName: \"kubernetes.io/projected/66939b12-acdb-4eb8-b426-ee87b793cf28-kube-api-access-8lgss\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.972707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.972671    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"]
Oct 30 19:30:20.978802 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:20.978772    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-441836-29889810-mtldb"]
Oct 30 19:30:21.495154 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.495126    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"
Oct 30 19:30:21.495154 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.495134    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc" event={"ID":"66939b12-acdb-4eb8-b426-ee87b793cf28","Type":"ContainerDied","Data":"8b1592be6af3a585e6908fb3703df9d5a608584f4cf747f6eef7e96e61a92beb"}
Oct 30 19:30:21.495682 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.495191    5232 scope.go:122] "RemoveContainer" containerID="2c0c9da2ec78b65f6c95ae80587cbca0408dcc621840fe080ed5154b1ea4e463"
Oct 30 19:30:21.566035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.565998    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" path="/var/lib/kubelet/pods/174636b6-19c0-44b9-8dd5-30a36ce8573f/volumes"
Oct 30 19:30:21.566674 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.566653    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" path="/var/lib/kubelet/pods/70922e73-0a54-4c22-b079-0a3b7538a22e/volumes"
Oct 30 19:30:21.668826 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.668785    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"]
Oct 30 19:30:21.673993 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.673962    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-2a5c13-29889810-8f2sc"]
Oct 30 19:30:21.709386 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.709352    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:30:21.709538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:21.709403    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:30:22.709881 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:22.709838    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:30:22.710316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:22.709898    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:30:23.565193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:23.565153    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" path="/var/lib/kubelet/pods/66939b12-acdb-4eb8-b426-ee87b793cf28/volumes"
Oct 30 19:30:23.710234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:23.710197    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:30:23.710635 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:23.710253    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:30:24.709766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:24.709721    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:30:24.710014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:24.709793    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:30:25.713979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:25.713945    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:30:25.714410 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:25.714249    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:30:25.714901 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:25.714883    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:30:38.635003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:38.634967    5232 generic.go:356] "Generic (PLEG): container finished" podID="43d1871f-405c-4db4-a424-4662b1e910cf" containerID="fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2" exitCode=0
Oct 30 19:30:38.635429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:38.635042    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" event={"ID":"43d1871f-405c-4db4-a424-4662b1e910cf","Type":"ContainerDied","Data":"fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2"}
Oct 30 19:30:38.635535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:38.635518    5232 scope.go:122] "RemoveContainer" containerID="fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2"
Oct 30 19:30:38.637000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:38.636971    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:38.637100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:38.637038    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:38.637100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:38.637072    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:30:39.668308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:39.668273    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" event={"ID":"43d1871f-405c-4db4-a424-4662b1e910cf","Type":"ContainerStarted","Data":"a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2"}
Oct 30 19:30:39.761916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:39.756186    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:30:39.762979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:39.762947    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 19:30:39.763074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:39.763012    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 19:30:40.759843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:40.756024    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 19:30:40.759843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:40.756088    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 19:30:41.767119 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:41.767080    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 19:30:41.767595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:41.767143    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 19:30:42.759013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:42.758972    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 19:30:42.759315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:42.759282    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 19:30:43.762976 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:43.762942    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 19:30:43.763431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:43.763003    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 19:30:44.754989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:44.754955    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 19:30:44.755204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:44.755014    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 19:30:45.655691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:45.655655    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:30:45.821399 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:45.821369    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:30:45.823211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:30:45.823187    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:30:46.188750 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:30:46.188717    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:31:05.084935 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:05.084899    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"]
Oct 30 19:31:05.085444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:05.085135    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy" containerID="cri-o://7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8" gracePeriod=30
Oct 30 19:31:10.353200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.353143    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:31:10.444073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444039    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-data\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444083    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-credential-socket\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444124    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-istio-token\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-istio-token\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444147    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-envoy\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444166    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-certs\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444271    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-socket\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444304    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-kube-api-access-v4wwj\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-kube-api-access-v4wwj\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444340    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-data" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:10.444467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444348    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50030bcc-3a51-420a-8542-fde4481596db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50030bcc-3a51-420a-8542-fde4481596db-istiod-ca-cert\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444426    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/50030bcc-3a51-420a-8542-fde4481596db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50030bcc-3a51-420a-8542-fde4481596db-istio-podinfo\") pod \"50030bcc-3a51-420a-8542-fde4481596db\" (UID: \"50030bcc-3a51-420a-8542-fde4481596db\") "
Oct 30 19:31:10.444735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444511    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-certs" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:10.444735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444714    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-socket" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:10.444872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444729    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50030bcc-3a51-420a-8542-fde4481596db-istiod-ca-cert" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:10.444932 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.444898    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-credential-socket" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:10.445038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.445012    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.445178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.445042    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50030bcc-3a51-420a-8542-fde4481596db-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.445178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.445056    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.445178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.445070    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.445178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.445085    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.446439 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.446413    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-envoy" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:10.446630 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.446608    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-kube-api-access-v4wwj" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "kube-api-access-v4wwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:10.446702 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.446683    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-istio-token" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:10.446984 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.446965    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50030bcc-3a51-420a-8542-fde4481596db-istio-podinfo" pod "50030bcc-3a51-420a-8542-fde4481596db" (UID: "50030bcc-3a51-420a-8542-fde4481596db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:31:10.546238 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.546190    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.546238 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.546225    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50030bcc-3a51-420a-8542-fde4481596db-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.546238 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.546242    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4wwj\" (UniqueName: \"kubernetes.io/projected/50030bcc-3a51-420a-8542-fde4481596db-kube-api-access-v4wwj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.546477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.546257    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50030bcc-3a51-420a-8542-fde4481596db-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:10.896532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.896493    5232 generic.go:356] "Generic (PLEG): container finished" podID="50030bcc-3a51-420a-8542-fde4481596db" containerID="7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8" exitCode=0
Oct 30 19:31:10.896740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.896536    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" event={"ID":"50030bcc-3a51-420a-8542-fde4481596db","Type":"ContainerDied","Data":"7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8"}
Oct 30 19:31:10.896740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.896546    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"
Oct 30 19:31:10.896740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.896591    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2" event={"ID":"50030bcc-3a51-420a-8542-fde4481596db","Type":"ContainerDied","Data":"7ae654114eaa4d8540710282bea0fc219f2afc394731ebda63d5163eb49ce99a"}
Oct 30 19:31:10.896740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.896620    5232 scope.go:122] "RemoveContainer" containerID="7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8"
Oct 30 19:31:10.907934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.907911    5232 scope.go:122] "RemoveContainer" containerID="111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396"
Oct 30 19:31:10.917073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.917047    5232 scope.go:122] "RemoveContainer" containerID="7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8"
Oct 30 19:31:10.917308 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:31:10.917289    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8\": container with ID starting with 7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8 not found: ID does not exist" containerID="7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8"
Oct 30 19:31:10.917352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.917318    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8"} err="failed to get container status \"7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8\": rpc error: code = NotFound desc = could not find container \"7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8\": container with ID starting with 7aee76d95d3c5beb1da8a37a4a240317282b716f7088e6344a0cdaebcdb186e8 not found: ID does not exist"
Oct 30 19:31:10.917352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.917343    5232 scope.go:122] "RemoveContainer" containerID="111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396"
Oct 30 19:31:10.917557 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:31:10.917538    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396\": container with ID starting with 111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396 not found: ID does not exist" containerID="111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396"
Oct 30 19:31:10.917595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:10.917562    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396"} err="failed to get container status \"111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396\": rpc error: code = NotFound desc = could not find container \"111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396\": container with ID starting with 111af93cfb2ec40e44cc0e9856a24a264148582fcd71df8a9f67fcdfb4683396 not found: ID does not exist"
Oct 30 19:31:11.056954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:11.056914    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"]
Oct 30 19:31:11.104556 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:11.104515    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-4833f3-openshift-default-6cd9888bdf-mfpp2"]
Oct 30 19:31:11.546190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:11.546158    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50030bcc-3a51-420a-8542-fde4481596db" path="/var/lib/kubelet/pods/50030bcc-3a51-420a-8542-fde4481596db/volumes"
Oct 30 19:31:16.188483 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:31:16.188441    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:31:34.938884 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:34.938855    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-9zk2f_528babed-84aa-491f-85a9-16cc40b8edc9/payload-processing/0.log"
Oct 30 19:31:35.210539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:35.210470    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-9zk2f_528babed-84aa-491f-85a9-16cc40b8edc9/payload-processing/0.log"
Oct 30 19:31:46.191322 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:31:46.191273    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:31:47.936184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.936155    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:31:47.940692 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.940673    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:31:47.955388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.955362    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:31:47.959420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.959381    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:31:47.985879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.985841    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"]
Oct 30 19:31:47.986631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986597    5232 cpu_manager.go:414] "containerMap: removing container" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.986631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986623    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986638    5232 cpu_manager.go:414] "containerMap: removing container" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986645    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986663    5232 cpu_manager.go:414] "containerMap: removing container" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986671    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986682    5232 cpu_manager.go:414] "containerMap: removing container" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986687    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986699    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986704    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986716    5232 cpu_manager.go:414] "containerMap: removing container" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986720    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986728    5232 cpu_manager.go:414] "containerMap: removing container" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986732    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986739    5232 cpu_manager.go:414] "containerMap: removing container" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986743    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986748    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986753    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986761    5232 cpu_manager.go:414] "containerMap: removing container" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986765    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986779    5232 cpu_manager.go:414] "containerMap: removing container" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986784    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986800    5232 cpu_manager.go:414] "containerMap: removing container" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986805    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986826    5232 cpu_manager.go:414] "containerMap: removing container" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986831    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986839    5232 cpu_manager.go:414] "containerMap: removing container" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:31:47.986841 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986844    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986932    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986940    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986947    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986953    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986959    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="66939b12-acdb-4eb8-b426-ee87b793cf28" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986965    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986972    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986979    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986986    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.986993    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.987004    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:31:47.987663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.987010    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:31:47.990228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:47.990208    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.041517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.041485    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-69d429-serving-cert\""
Oct 30 19:31:48.092024 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.091991    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"]
Oct 30 19:31:48.124502 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.124462    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/872dc425-c1d5-45d8-99f8-e62d6e67f54a-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.124699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.124661    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-694nx\" (UniqueName: \"kubernetes.io/projected/872dc425-c1d5-45d8-99f8-e62d6e67f54a-kube-api-access-694nx\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.124782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.124717    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.124926 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.124901    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-metrics-certs\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.237909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.236865    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-metrics-certs\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.237909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.237014    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/872dc425-c1d5-45d8-99f8-e62d6e67f54a-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.237909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.237172    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-694nx\" (UniqueName: \"kubernetes.io/projected/872dc425-c1d5-45d8-99f8-e62d6e67f54a-kube-api-access-694nx\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.237909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.237232    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.237909 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:31:48.237460    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-serving-cert: secret "maas-api-e2e-praxis-b-69d429-serving-cert" not found
Oct 30 19:31:48.237909 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:31:48.237538    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls podName:872dc425-c1d5-45d8-99f8-e62d6e67f54a nodeName:}" failed. No retries permitted until 2026-10-30 19:31:48.73751669 +0000 UTC m=+2701.321999677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls") pod "maas-api-e2e-praxis-b-69d429-b57878656-qxsmh" (UID: "872dc425-c1d5-45d8-99f8-e62d6e67f54a") : secret "maas-api-e2e-praxis-b-69d429-serving-cert" not found
Oct 30 19:31:48.239374 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.239342    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/872dc425-c1d5-45d8-99f8-e62d6e67f54a-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.241730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.241702    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-metrics-certs\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.304780 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.304749    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-694nx\" (UniqueName: \"kubernetes.io/projected/872dc425-c1d5-45d8-99f8-e62d6e67f54a-kube-api-access-694nx\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.742456 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.742415    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.744857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.744825    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls\") pod \"maas-api-e2e-praxis-b-69d429-b57878656-qxsmh\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:48.904760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:48.904729    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:49.284025 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:49.283937    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"]
Oct 30 19:31:49.288366 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:31:49.288338    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872dc425_c1d5_45d8_99f8_e62d6e67f54a.slice/crio-288e483ea9f4e016f2c8c10d8d17b47b0b8d66cc78ea8a536671b717d80c64fe WatchSource:0}: Error finding container 288e483ea9f4e016f2c8c10d8d17b47b0b8d66cc78ea8a536671b717d80c64fe: Status 404 returned error can't find the container with id 288e483ea9f4e016f2c8c10d8d17b47b0b8d66cc78ea8a536671b717d80c64fe
Oct 30 19:31:50.161136 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:50.161102    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh" event={"ID":"872dc425-c1d5-45d8-99f8-e62d6e67f54a","Type":"ContainerStarted","Data":"288e483ea9f4e016f2c8c10d8d17b47b0b8d66cc78ea8a536671b717d80c64fe"}
Oct 30 19:31:51.169585 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:51.169544    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh" event={"ID":"872dc425-c1d5-45d8-99f8-e62d6e67f54a","Type":"ContainerStarted","Data":"2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377"}
Oct 30 19:31:51.170071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:51.169760    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:31:51.286734 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:51.286673    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh" podStartSLOduration=3.448256813 podStartE2EDuration="4.286660472s" totalImagesPullingTime="838.403659ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:51.283770718 +0000 UTC m=+2703.868253728" watchObservedRunningTime="2026-10-30 19:31:51.286660472 +0000 UTC m=+2703.871143468"
Oct 30 19:31:57.181350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:31:57.181268    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:32:06.148723 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.148683    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"]
Oct 30 19:32:06.149424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.148940    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj" podUID="22d77107-0992-41b9-adfc-2e1f0826a477" containerName="maas-api" containerID="cri-o://725f64d34302c2daff1eb50768d0d4255f77af8ecc4c43ffa49f77199156c413" gracePeriod=30
Oct 30 19:32:06.270627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.270600    5232 generic.go:356] "Generic (PLEG): container finished" podID="22d77107-0992-41b9-adfc-2e1f0826a477" containerID="725f64d34302c2daff1eb50768d0d4255f77af8ecc4c43ffa49f77199156c413" exitCode=0
Oct 30 19:32:06.270731 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.270668    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj" event={"ID":"22d77107-0992-41b9-adfc-2e1f0826a477","Type":"ContainerDied","Data":"725f64d34302c2daff1eb50768d0d4255f77af8ecc4c43ffa49f77199156c413"}
Oct 30 19:32:06.453899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.453857    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:32:06.568627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.568593    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-metrics-certs\") pod \"22d77107-0992-41b9-adfc-2e1f0826a477\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") "
Oct 30 19:32:06.568860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.568744    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22d77107-0992-41b9-adfc-2e1f0826a477-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d77107-0992-41b9-adfc-2e1f0826a477-openshift-service-ca\") pod \"22d77107-0992-41b9-adfc-2e1f0826a477\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") "
Oct 30 19:32:06.568949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.568857    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls\") pod \"22d77107-0992-41b9-adfc-2e1f0826a477\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") "
Oct 30 19:32:06.568949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.568912    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22d77107-0992-41b9-adfc-2e1f0826a477-kube-api-access-wlxw2\" (UniqueName: \"kubernetes.io/projected/22d77107-0992-41b9-adfc-2e1f0826a477-kube-api-access-wlxw2\") pod \"22d77107-0992-41b9-adfc-2e1f0826a477\" (UID: \"22d77107-0992-41b9-adfc-2e1f0826a477\") "
Oct 30 19:32:06.569201 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.569168    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d77107-0992-41b9-adfc-2e1f0826a477-openshift-service-ca" pod "22d77107-0992-41b9-adfc-2e1f0826a477" (UID: "22d77107-0992-41b9-adfc-2e1f0826a477"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:06.569454 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.569427    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22d77107-0992-41b9-adfc-2e1f0826a477-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:06.571111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.571083    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls" pod "22d77107-0992-41b9-adfc-2e1f0826a477" (UID: "22d77107-0992-41b9-adfc-2e1f0826a477"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:06.571336 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.571305    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-metrics-certs" pod "22d77107-0992-41b9-adfc-2e1f0826a477" (UID: "22d77107-0992-41b9-adfc-2e1f0826a477"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:06.571336 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.571317    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d77107-0992-41b9-adfc-2e1f0826a477-kube-api-access-wlxw2" pod "22d77107-0992-41b9-adfc-2e1f0826a477" (UID: "22d77107-0992-41b9-adfc-2e1f0826a477"). InnerVolumeSpecName "kube-api-access-wlxw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:06.671293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.671215    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:06.671293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.671268    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlxw2\" (UniqueName: \"kubernetes.io/projected/22d77107-0992-41b9-adfc-2e1f0826a477-kube-api-access-wlxw2\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:06.671293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:06.671279    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22d77107-0992-41b9-adfc-2e1f0826a477-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:07.307431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:07.306609    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj" event={"ID":"22d77107-0992-41b9-adfc-2e1f0826a477","Type":"ContainerDied","Data":"14e10a5131f9a8a5227a0dbfb7d5e63c8bd255d5704c75213bf1ac080324bc87"}
Oct 30 19:32:07.307431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:07.306690    5232 scope.go:122] "RemoveContainer" containerID="725f64d34302c2daff1eb50768d0d4255f77af8ecc4c43ffa49f77199156c413"
Oct 30 19:32:07.307431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:07.306870    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"
Oct 30 19:32:07.468444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:07.468404    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"]
Oct 30 19:32:07.515637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:07.515602    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c16ba7-79769584dd-7k5hj"]
Oct 30 19:32:07.540962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:07.540930    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22d77107-0992-41b9-adfc-2e1f0826a477" path="/var/lib/kubelet/pods/22d77107-0992-41b9-adfc-2e1f0826a477/volumes"
Oct 30 19:32:12.748800 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.748748    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"]
Oct 30 19:32:12.749604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749585    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:32:12.749654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749606    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:32:12.749654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749626    5232 cpu_manager.go:414] "containerMap: removing container" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:32:12.749654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749631    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:32:12.749752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749656    5232 cpu_manager.go:414] "containerMap: removing container" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:32:12.749752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749661    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:32:12.749752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749675    5232 cpu_manager.go:414] "containerMap: removing container" podUID="22d77107-0992-41b9-adfc-2e1f0826a477" containerName="maas-api"
Oct 30 19:32:12.749752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749679    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22d77107-0992-41b9-adfc-2e1f0826a477" containerName="maas-api"
Oct 30 19:32:12.749910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749756    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="eae7d006-10ba-4f0c-91fb-f4bfe6a97419" containerName="cleanup"
Oct 30 19:32:12.749910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749769    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="22d77107-0992-41b9-adfc-2e1f0826a477" containerName="maas-api"
Oct 30 19:32:12.749910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749776    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="50030bcc-3a51-420a-8542-fde4481596db" containerName="istio-proxy"
Oct 30 19:32:12.749910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749783    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="174636b6-19c0-44b9-8dd5-30a36ce8573f" containerName="cleanup"
Oct 30 19:32:12.749910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749788    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9224ff9-b932-4c67-919c-69def5c59e67" containerName="cleanup"
Oct 30 19:32:12.749910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.749794    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="70922e73-0a54-4c22-b079-0a3b7538a22e" containerName="cleanup"
Oct 30 19:32:12.754496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.754476    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.804977 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.804945    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-e73f1a-serving-cert\""
Oct 30 19:32:12.848356 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.848318    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.848561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.848517    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q9mp\" (UniqueName: \"kubernetes.io/projected/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-kube-api-access-9q9mp\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.848637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.848598    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-metrics-certs\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.848695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.848639    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.856055 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.856027    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"]
Oct 30 19:32:12.949960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.949927    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.950140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.950005    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q9mp\" (UniqueName: \"kubernetes.io/projected/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-kube-api-access-9q9mp\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.950140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.950028    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-metrics-certs\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.950140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.950048    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.950276 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:32:12.950235    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-serving-cert: secret "maas-api-e2e-praxis-b-e73f1a-serving-cert" not found
Oct 30 19:32:12.950320 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:32:12.950309    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls podName:8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f nodeName:}" failed. No retries permitted until 2026-10-30 19:32:13.450288703 +0000 UTC m=+2726.034771691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls") pod "maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs" (UID: "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f") : secret "maas-api-e2e-praxis-b-e73f1a-serving-cert" not found
Oct 30 19:32:12.950739 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.950711    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:12.952556 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:12.952532    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-metrics-certs\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:13.011645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:13.011568    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q9mp\" (UniqueName: \"kubernetes.io/projected/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-kube-api-access-9q9mp\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:13.456340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:13.456306    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:13.458702 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:13.458673    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:13.668022 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:13.667980    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:14.044050 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:14.043972    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"]
Oct 30 19:32:14.049488 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:32:14.049449    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eab9c9e_4864_4f6e_89eb_b0d4ee4ff12f.slice/crio-29e331215c302fdf259dc8d0e7160328dce3c4b15df817c0834cda49e7f3e570 WatchSource:0}: Error finding container 29e331215c302fdf259dc8d0e7160328dce3c4b15df817c0834cda49e7f3e570: Status 404 returned error can't find the container with id 29e331215c302fdf259dc8d0e7160328dce3c4b15df817c0834cda49e7f3e570
Oct 30 19:32:14.353435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:14.353396    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs" event={"ID":"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f","Type":"ContainerStarted","Data":"29e331215c302fdf259dc8d0e7160328dce3c4b15df817c0834cda49e7f3e570"}
Oct 30 19:32:15.371230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:15.371182    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs" event={"ID":"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f","Type":"ContainerStarted","Data":"f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a"}
Oct 30 19:32:15.371624 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:15.371286    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:15.489925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:15.489604    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs" podStartSLOduration=2.572932329 podStartE2EDuration="3.489586608s" totalImagesPullingTime="916.654279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:15.486345819 +0000 UTC m=+2728.070828810" watchObservedRunningTime="2026-10-30 19:32:15.489586608 +0000 UTC m=+2728.074069671"
Oct 30 19:32:15.681753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:15.681623    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"]
Oct 30 19:32:15.681998 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:15.681972    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy" containerID="cri-o://e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee" gracePeriod=30
Oct 30 19:32:16.187348 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:32:16.187310    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:32:20.957118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:20.957052    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:32:21.071917 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.071883    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-envoy\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.071930    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-token\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-token\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.071954    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-kube-api-access-rnwl5\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-kube-api-access-rnwl5\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.071996    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-podinfo\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.072036    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-credential-socket\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.072125    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-socket\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.072176    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istiod-ca-cert\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.072277    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-certs\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072458 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.072358    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-data\") pod \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\" (UID: \"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c\") "
Oct 30 19:32:21.072518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.072496    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-socket" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.074360 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.073162    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-data" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.074360 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.073431    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-certs" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.074360 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.073534    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-credential-socket" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.074360 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.073888    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istiod-ca-cert" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:21.074639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.074550    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.074639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.074588    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.074639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.074604    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.074639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.074619    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.074766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.074643    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-podinfo" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:32:21.083694 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.083425    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-kube-api-access-rnwl5" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "kube-api-access-rnwl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:21.083694 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.083516    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-envoy" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.083694 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.083517    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-token" pod "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" (UID: "68859be2-cf72-46a2-bc8a-5d7ba22c0a4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:21.175467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.175429    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.175467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.175461    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.175467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.175474    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnwl5\" (UniqueName: \"kubernetes.io/projected/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-kube-api-access-rnwl5\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.175722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.175484    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.175722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.175494    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.383042 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.383014    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:32:21.411568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.411534    5232 generic.go:356] "Generic (PLEG): container finished" podID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerID="e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee" exitCode=0
Oct 30 19:32:21.411777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.411595    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"
Oct 30 19:32:21.411777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.411616    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" event={"ID":"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c","Type":"ContainerDied","Data":"e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee"}
Oct 30 19:32:21.411777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.411673    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs" event={"ID":"68859be2-cf72-46a2-bc8a-5d7ba22c0a4c","Type":"ContainerDied","Data":"fa621c5db0d7d63a35bab7d6831ed8f80c1b37e6ba0369a6333eed8a5bb61885"}
Oct 30 19:32:21.411777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.411696    5232 scope.go:122] "RemoveContainer" containerID="e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee"
Oct 30 19:32:21.422189 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.422168    5232 scope.go:122] "RemoveContainer" containerID="90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb"
Oct 30 19:32:21.431562 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.431541    5232 scope.go:122] "RemoveContainer" containerID="e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee"
Oct 30 19:32:21.431832 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:32:21.431795    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee\": container with ID starting with e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee not found: ID does not exist" containerID="e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee"
Oct 30 19:32:21.431905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.431843    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee"} err="failed to get container status \"e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee\": rpc error: code = NotFound desc = could not find container \"e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee\": container with ID starting with e35915355ab37e29a3faaefbd1f0d9010f46e68ab1249e4b7338887fb52872ee not found: ID does not exist"
Oct 30 19:32:21.431905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.431867    5232 scope.go:122] "RemoveContainer" containerID="90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb"
Oct 30 19:32:21.432142 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:32:21.432123    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb\": container with ID starting with 90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb not found: ID does not exist" containerID="90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb"
Oct 30 19:32:21.432183 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.432150    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb"} err="failed to get container status \"90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb\": rpc error: code = NotFound desc = could not find container \"90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb\": container with ID starting with 90121f15f545744515726ba2233153fb8a45771cca9f5877391334b37d79d6bb not found: ID does not exist"
Oct 30 19:32:21.615280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.615077    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"]
Oct 30 19:32:21.639947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:21.633436    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-c16ba7-openshift-default-f6b789b57-29gjs"]
Oct 30 19:32:23.539638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:23.539601    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" path="/var/lib/kubelet/pods/68859be2-cf72-46a2-bc8a-5d7ba22c0a4c/volumes"
Oct 30 19:32:39.040946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.040908    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"]
Oct 30 19:32:39.041689 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.041666    5232 cpu_manager.go:414] "containerMap: removing container" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy"
Oct 30 19:32:39.041792 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.041691    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy"
Oct 30 19:32:39.041792 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.041713    5232 cpu_manager.go:414] "containerMap: removing container" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy"
Oct 30 19:32:39.041792 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.041720    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy"
Oct 30 19:32:39.041974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.041896    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy"
Oct 30 19:32:39.045610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.045588    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.098305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.098266    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-83ac32-openshift-default-dockercfg-mpq86\""
Oct 30 19:32:39.148229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.148198    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"]
Oct 30 19:32:39.237608 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.237561    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-credential-socket\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.237919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.237893    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-envoy\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.238122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.238104    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-token\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.239649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.238274    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-socket\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.239944 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.239919    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-podinfo\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.240192 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.240171    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-certs\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.240383 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.240362    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xmng\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-kube-api-access-9xmng\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.240574 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.240556    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istiod-ca-cert\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.240695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.240677    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-data\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.341755 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.341707    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-token\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.341976 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.341769    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-socket\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.341976 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.341865    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-podinfo\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.341976 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.341905    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-certs\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.341976 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.341939    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xmng\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-kube-api-access-9xmng\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.342225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.341977    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istiod-ca-cert\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.342225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.342010    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-data\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.342225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.342138    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-credential-socket\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.342225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.342175    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-envoy\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.342402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.342254    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-socket\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.342672 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.342647    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-certs\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.342895 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.342868    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-credential-socket\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.343080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.342911    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istiod-ca-cert\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.343080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.342957    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-data\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.344730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.344709    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-podinfo\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.344916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.344894    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-envoy\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.401125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.401096    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-token\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.401125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.401123    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xmng\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-kube-api-access-9xmng\") pod \"e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") " pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:39.659052 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:39.658973    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:40.089545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:40.089515    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"]
Oct 30 19:32:40.128439 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:32:40.128406    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504ff7c3_69c5_4422_a4b2_ba43ecf1197f.slice/crio-2bbd11e3036b28c21196c4918e7391f50c6fbb03158181c760aac8c616372901 WatchSource:0}: Error finding container 2bbd11e3036b28c21196c4918e7391f50c6fbb03158181c760aac8c616372901: Status 404 returned error can't find the container with id 2bbd11e3036b28c21196c4918e7391f50c6fbb03158181c760aac8c616372901
Oct 30 19:32:40.130879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:40.130841    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:40.130954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:40.130914    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:40.130954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:40.130946    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:40.559111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:40.559008    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" event={"ID":"504ff7c3-69c5-4422-a4b2-ba43ecf1197f","Type":"ContainerStarted","Data":"bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13"}
Oct 30 19:32:40.559111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:40.559065    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" event={"ID":"504ff7c3-69c5-4422-a4b2-ba43ecf1197f","Type":"ContainerStarted","Data":"2bbd11e3036b28c21196c4918e7391f50c6fbb03158181c760aac8c616372901"}
Oct 30 19:32:40.659688 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:40.659656    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:40.723964 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:40.723912    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" podStartSLOduration=2.723898307 podStartE2EDuration="2.723898307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:40.722450901 +0000 UTC m=+2753.306933899" watchObservedRunningTime="2026-10-30 19:32:40.723898307 +0000 UTC m=+2753.308381303"
Oct 30 19:32:41.665490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:41.665460    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:42.571642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:42.571607    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:42.572603 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:32:42.572583    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:32:46.187251 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:32:46.187217    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:33:16.187073 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:33:16.187031    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:33:31.918974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:31.918884    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"]
Oct 30 19:33:32.021974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:32.021936    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"]
Oct 30 19:33:36.919397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:36.919349    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8" podUID="acf82f67-3d24-460e-b7bd-a0b378b7452d" containerName="payload-processing" containerID="cri-o://cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c" gracePeriod=25
Oct 30 19:33:37.022947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.022908    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9" podUID="f0609fae-2afd-4751-9f61-0fb155029862" containerName="payload-pre-processing" containerID="cri-o://289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7" gracePeriod=25
Oct 30 19:33:37.191178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.191149    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:33:37.265948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.265913    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acf82f67-3d24-460e-b7bd-a0b378b7452d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf82f67-3d24-460e-b7bd-a0b378b7452d-plugins-config-volume\") pod \"acf82f67-3d24-460e-b7bd-a0b378b7452d\" (UID: \"acf82f67-3d24-460e-b7bd-a0b378b7452d\") "
Oct 30 19:33:37.266128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.266096    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acf82f67-3d24-460e-b7bd-a0b378b7452d-kube-api-access-wxzdj\" (UniqueName: \"kubernetes.io/projected/acf82f67-3d24-460e-b7bd-a0b378b7452d-kube-api-access-wxzdj\") pod \"acf82f67-3d24-460e-b7bd-a0b378b7452d\" (UID: \"acf82f67-3d24-460e-b7bd-a0b378b7452d\") "
Oct 30 19:33:37.266312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.266283    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf82f67-3d24-460e-b7bd-a0b378b7452d-plugins-config-volume" pod "acf82f67-3d24-460e-b7bd-a0b378b7452d" (UID: "acf82f67-3d24-460e-b7bd-a0b378b7452d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:37.266561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.266544    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf82f67-3d24-460e-b7bd-a0b378b7452d-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:37.268310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.268284    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf82f67-3d24-460e-b7bd-a0b378b7452d-kube-api-access-wxzdj" pod "acf82f67-3d24-460e-b7bd-a0b378b7452d" (UID: "acf82f67-3d24-460e-b7bd-a0b378b7452d"). InnerVolumeSpecName "kube-api-access-wxzdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:37.288616 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.288593    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:33:37.367933 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.367899    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0609fae-2afd-4751-9f61-0fb155029862-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0609fae-2afd-4751-9f61-0fb155029862-plugins-config-volume\") pod \"f0609fae-2afd-4751-9f61-0fb155029862\" (UID: \"f0609fae-2afd-4751-9f61-0fb155029862\") "
Oct 30 19:33:37.368105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.368033    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0609fae-2afd-4751-9f61-0fb155029862-kube-api-access-mnlkt\" (UniqueName: \"kubernetes.io/projected/f0609fae-2afd-4751-9f61-0fb155029862-kube-api-access-mnlkt\") pod \"f0609fae-2afd-4751-9f61-0fb155029862\" (UID: \"f0609fae-2afd-4751-9f61-0fb155029862\") "
Oct 30 19:33:37.368268 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.368245    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0609fae-2afd-4751-9f61-0fb155029862-plugins-config-volume" pod "f0609fae-2afd-4751-9f61-0fb155029862" (UID: "f0609fae-2afd-4751-9f61-0fb155029862"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:37.368385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.368371    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0609fae-2afd-4751-9f61-0fb155029862-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:37.368442 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.368387    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxzdj\" (UniqueName: \"kubernetes.io/projected/acf82f67-3d24-460e-b7bd-a0b378b7452d-kube-api-access-wxzdj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:37.370034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.370015    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0609fae-2afd-4751-9f61-0fb155029862-kube-api-access-mnlkt" pod "f0609fae-2afd-4751-9f61-0fb155029862" (UID: "f0609fae-2afd-4751-9f61-0fb155029862"). InnerVolumeSpecName "kube-api-access-mnlkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:37.469766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:37.469685    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnlkt\" (UniqueName: \"kubernetes.io/projected/f0609fae-2afd-4751-9f61-0fb155029862-kube-api-access-mnlkt\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:38.009910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.009871    5232 generic.go:356] "Generic (PLEG): container finished" podID="f0609fae-2afd-4751-9f61-0fb155029862" containerID="289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7" exitCode=0
Oct 30 19:33:38.010380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.009945    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"
Oct 30 19:33:38.010380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.009970    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9" event={"ID":"f0609fae-2afd-4751-9f61-0fb155029862","Type":"ContainerDied","Data":"289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7"}
Oct 30 19:33:38.010380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.010112    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9" event={"ID":"f0609fae-2afd-4751-9f61-0fb155029862","Type":"ContainerDied","Data":"cff53809a7be5080ce154695aa8a353e4d5edc83563a19689e42d300b785becc"}
Oct 30 19:33:38.010380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.010159    5232 scope.go:122] "RemoveContainer" containerID="289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7"
Oct 30 19:33:38.011583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.011518    5232 generic.go:356] "Generic (PLEG): container finished" podID="acf82f67-3d24-460e-b7bd-a0b378b7452d" containerID="cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c" exitCode=0
Oct 30 19:33:38.011706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.011688    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8" event={"ID":"acf82f67-3d24-460e-b7bd-a0b378b7452d","Type":"ContainerDied","Data":"cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c"}
Oct 30 19:33:38.011779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.011732    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8" event={"ID":"acf82f67-3d24-460e-b7bd-a0b378b7452d","Type":"ContainerDied","Data":"018848cc6a4bf33f70de4d0f52c99c58f0345cfeb8fde6448da6ccbbf9c61ad8"}
Oct 30 19:33:38.011867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.011851    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"
Oct 30 19:33:38.020769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.020752    5232 scope.go:122] "RemoveContainer" containerID="289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7"
Oct 30 19:33:38.021098 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:33:38.021071    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7\": container with ID starting with 289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7 not found: ID does not exist" containerID="289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7"
Oct 30 19:33:38.021184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.021107    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7"} err="failed to get container status \"289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7\": rpc error: code = NotFound desc = could not find container \"289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7\": container with ID starting with 289d95ad9c8161d69a2ec47494e11ccf77fc61f732a2eefdc76a10634b623ed7 not found: ID does not exist"
Oct 30 19:33:38.021184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.021130    5232 scope.go:122] "RemoveContainer" containerID="cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c"
Oct 30 19:33:38.030228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.030207    5232 scope.go:122] "RemoveContainer" containerID="cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c"
Oct 30 19:33:38.030510 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:33:38.030491    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c\": container with ID starting with cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c not found: ID does not exist" containerID="cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c"
Oct 30 19:33:38.030575 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.030522    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c"} err="failed to get container status \"cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c\": rpc error: code = NotFound desc = could not find container \"cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c\": container with ID starting with cfe0decc13fa330006c0518a4c25f0e6a90f90a96d7d0c9f09c26ce4c328977c not found: ID does not exist"
Oct 30 19:33:38.170359 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.170326    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"]
Oct 30 19:33:38.173026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.173004    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f9a20d-cb7b9d644-96vw8"]
Oct 30 19:33:38.276744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.276649    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"]
Oct 30 19:33:38.283403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:38.283374    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-f9a20d-7fb7f78b87-2t8x9"]
Oct 30 19:33:39.540596 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:39.540565    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acf82f67-3d24-460e-b7bd-a0b378b7452d" path="/var/lib/kubelet/pods/acf82f67-3d24-460e-b7bd-a0b378b7452d/volumes"
Oct 30 19:33:39.541034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:39.540988    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0609fae-2afd-4751-9f61-0fb155029862" path="/var/lib/kubelet/pods/f0609fae-2afd-4751-9f61-0fb155029862/volumes"
Oct 30 19:33:41.783788 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:41.783757    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"]
Oct 30 19:33:41.784265 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:41.784031    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh" podUID="bc40b3f2-ce90-4b44-ba94-6119d3b2e122" containerName="maas-api" containerID="cri-o://2702dedea0fa0811530f6a670b78ca58df8928a7a4c1d38c6ef1e6b9eae7459b" gracePeriod=30
Oct 30 19:33:42.041232 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.041200    5232 generic.go:356] "Generic (PLEG): container finished" podID="bc40b3f2-ce90-4b44-ba94-6119d3b2e122" containerID="2702dedea0fa0811530f6a670b78ca58df8928a7a4c1d38c6ef1e6b9eae7459b" exitCode=0
Oct 30 19:33:42.041367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.041293    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh" event={"ID":"bc40b3f2-ce90-4b44-ba94-6119d3b2e122","Type":"ContainerDied","Data":"2702dedea0fa0811530f6a670b78ca58df8928a7a4c1d38c6ef1e6b9eae7459b"}
Oct 30 19:33:42.068332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.068299    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:33:42.124031 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.124000    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-maas-api-tls\") pod \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") "
Oct 30 19:33:42.124197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.124110    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-openshift-service-ca\") pod \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") "
Oct 30 19:33:42.124197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.124178    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-metrics-certs\") pod \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") "
Oct 30 19:33:42.124277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.124223    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-kube-api-access-nwzmv\" (UniqueName: \"kubernetes.io/projected/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-kube-api-access-nwzmv\") pod \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\" (UID: \"bc40b3f2-ce90-4b44-ba94-6119d3b2e122\") "
Oct 30 19:33:42.124478 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.124448    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-openshift-service-ca" pod "bc40b3f2-ce90-4b44-ba94-6119d3b2e122" (UID: "bc40b3f2-ce90-4b44-ba94-6119d3b2e122"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:42.124695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.124678    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:42.126223 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.126195    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-maas-api-tls" pod "bc40b3f2-ce90-4b44-ba94-6119d3b2e122" (UID: "bc40b3f2-ce90-4b44-ba94-6119d3b2e122"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:42.126313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.126219    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-metrics-certs" pod "bc40b3f2-ce90-4b44-ba94-6119d3b2e122" (UID: "bc40b3f2-ce90-4b44-ba94-6119d3b2e122"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:42.126313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.126294    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-kube-api-access-nwzmv" pod "bc40b3f2-ce90-4b44-ba94-6119d3b2e122" (UID: "bc40b3f2-ce90-4b44-ba94-6119d3b2e122"). InnerVolumeSpecName "kube-api-access-nwzmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:42.225443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.225406    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:42.225443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.225441    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:42.225649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:42.225452    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwzmv\" (UniqueName: \"kubernetes.io/projected/bc40b3f2-ce90-4b44-ba94-6119d3b2e122-kube-api-access-nwzmv\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:43.049068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:43.049038    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"
Oct 30 19:33:43.049611 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:43.049079    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh" event={"ID":"bc40b3f2-ce90-4b44-ba94-6119d3b2e122","Type":"ContainerDied","Data":"d6e744ed3a3faf5b5f0d6a99de3ac68a9bb9516345c395266ab0a01799cb3797"}
Oct 30 19:33:43.049611 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:43.049149    5232 scope.go:122] "RemoveContainer" containerID="2702dedea0fa0811530f6a670b78ca58df8928a7a4c1d38c6ef1e6b9eae7459b"
Oct 30 19:33:43.209289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:43.209254    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"]
Oct 30 19:33:43.256671 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:43.256636    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f9a20d-7dd548f9df-7jbzh"]
Oct 30 19:33:43.541020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:43.540986    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc40b3f2-ce90-4b44-ba94-6119d3b2e122" path="/var/lib/kubelet/pods/bc40b3f2-ce90-4b44-ba94-6119d3b2e122/volumes"
Oct 30 19:33:46.187755 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:33:46.187722    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:33:47.209096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:47.209057    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"]
Oct 30 19:33:47.209535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:47.209345    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy" containerID="cri-o://e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd" gracePeriod=30
Oct 30 19:33:49.696038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:49.696001    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:33:49.696491 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:49.696075    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:33:52.463928 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.463898    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:33:52.548859 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.548757    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-credential-socket\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.548866    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-podinfo\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.548902    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c908ea0-f632-4c83-b6c9-440f60a50c99-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c908ea0-f632-4c83-b6c9-440f60a50c99-istiod-ca-cert\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.548937    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-kube-api-access-5fh2t\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-kube-api-access-5fh2t\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.548958    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-token\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-token\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.548982    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-data\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549050    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-envoy\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549061    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-credential-socket" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:52.549319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549121    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-certs\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549246    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-socket\") pod \"1c908ea0-f632-4c83-b6c9-440f60a50c99\" (UID: \"1c908ea0-f632-4c83-b6c9-440f60a50c99\") "
Oct 30 19:33:52.549319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549287    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-data" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:52.549562 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549358    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c908ea0-f632-4c83-b6c9-440f60a50c99-istiod-ca-cert" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:52.549562 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549510    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-certs" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:52.549656 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549630    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-socket" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:52.549893 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549870    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c908ea0-f632-4c83-b6c9-440f60a50c99-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.549971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549899    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.549971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549914    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.549971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549930    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.549971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.549945    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.551197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.551169    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-token" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:52.551299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.551224    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-podinfo" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:52.551542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.551518    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-kube-api-access-5fh2t" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "kube-api-access-5fh2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:52.551542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.551522    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-envoy" pod "1c908ea0-f632-4c83-b6c9-440f60a50c99" (UID: "1c908ea0-f632-4c83-b6c9-440f60a50c99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:52.651433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.651392    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.651433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.651422    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fh2t\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-kube-api-access-5fh2t\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.651433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.651434    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.651433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.651442    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c908ea0-f632-4c83-b6c9-440f60a50c99-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:52.912773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.912740    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"]
Oct 30 19:33:52.913519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913490    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f0609fae-2afd-4751-9f61-0fb155029862" containerName="payload-pre-processing"
Oct 30 19:33:52.913519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913517    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0609fae-2afd-4751-9f61-0fb155029862" containerName="payload-pre-processing"
Oct 30 19:33:52.913712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913551    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy"
Oct 30 19:33:52.913712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913560    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy"
Oct 30 19:33:52.913712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913572    5232 cpu_manager.go:414] "containerMap: removing container" podUID="bc40b3f2-ce90-4b44-ba94-6119d3b2e122" containerName="maas-api"
Oct 30 19:33:52.913712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913580    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc40b3f2-ce90-4b44-ba94-6119d3b2e122" containerName="maas-api"
Oct 30 19:33:52.913712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913616    5232 cpu_manager.go:414] "containerMap: removing container" podUID="acf82f67-3d24-460e-b7bd-a0b378b7452d" containerName="payload-processing"
Oct 30 19:33:52.913712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913623    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acf82f67-3d24-460e-b7bd-a0b378b7452d" containerName="payload-processing"
Oct 30 19:33:52.913712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913642    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy"
Oct 30 19:33:52.913712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913649    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy"
Oct 30 19:33:52.914035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913744    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy"
Oct 30 19:33:52.914035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913755    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc40b3f2-ce90-4b44-ba94-6119d3b2e122" containerName="maas-api"
Oct 30 19:33:52.914035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913771    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="acf82f67-3d24-460e-b7bd-a0b378b7452d" containerName="payload-processing"
Oct 30 19:33:52.914035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913781    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="68859be2-cf72-46a2-bc8a-5d7ba22c0a4c" containerName="istio-proxy"
Oct 30 19:33:52.914035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.913788    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0609fae-2afd-4751-9f61-0fb155029862" containerName="payload-pre-processing"
Oct 30 19:33:52.917347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.917328    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:52.968027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.967992    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-83ac32-serving-cert\""
Oct 30 19:33:52.977334 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:52.977302    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"]
Oct 30 19:33:53.055340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.055306    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.055534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.055388    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-metrics-certs\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.055534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.055499    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzh52\" (UniqueName: \"kubernetes.io/projected/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-kube-api-access-pzh52\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.055621 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.055584    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.121453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.121415    5232 generic.go:356] "Generic (PLEG): container finished" podID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerID="e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd" exitCode=0
Oct 30 19:33:53.121631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.121470    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"
Oct 30 19:33:53.121631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.121501    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" event={"ID":"1c908ea0-f632-4c83-b6c9-440f60a50c99","Type":"ContainerDied","Data":"e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd"}
Oct 30 19:33:53.121631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.121560    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm" event={"ID":"1c908ea0-f632-4c83-b6c9-440f60a50c99","Type":"ContainerDied","Data":"69087a6abdbd9b2b4bab242c4cf0d29be3070d35026760eee48183b89b894a97"}
Oct 30 19:33:53.121631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.121593    5232 scope.go:122] "RemoveContainer" containerID="e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd"
Oct 30 19:33:53.131743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.131725    5232 scope.go:122] "RemoveContainer" containerID="1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267"
Oct 30 19:33:53.143693 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.143668    5232 scope.go:122] "RemoveContainer" containerID="e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd"
Oct 30 19:33:53.144292 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:33:53.144267    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd\": container with ID starting with e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd not found: ID does not exist" containerID="e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd"
Oct 30 19:33:53.144401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.144306    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd"} err="failed to get container status \"e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd\": rpc error: code = NotFound desc = could not find container \"e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd\": container with ID starting with e51ad62560c5de0f2b57c51ef5575892e7a2782f6f1a01ea735dc0dd147ffbcd not found: ID does not exist"
Oct 30 19:33:53.144401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.144333    5232 scope.go:122] "RemoveContainer" containerID="1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267"
Oct 30 19:33:53.146672 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:33:53.146641    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267\": container with ID starting with 1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267 not found: ID does not exist" containerID="1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267"
Oct 30 19:33:53.146753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.146684    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267"} err="failed to get container status \"1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267\": rpc error: code = NotFound desc = could not find container \"1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267\": container with ID starting with 1081578e3a8366831c85d2a7249a17c77d3d0bf4494c0e72a39812a3635e5267 not found: ID does not exist"
Oct 30 19:33:53.156508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.156471    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.156661 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.156577    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.156734 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.156660    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-metrics-certs\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.156793 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.156765    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzh52\" (UniqueName: \"kubernetes.io/projected/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-kube-api-access-pzh52\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.157343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.157321    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.159359 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.159337    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-metrics-certs\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.159559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.159534    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.216327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.216253    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzh52\" (UniqueName: \"kubernetes.io/projected/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-kube-api-access-pzh52\") pod \"maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.231063 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.231035    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:53.281923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.281876    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"]
Oct 30 19:33:53.299674 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.299251    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-7a790b-openshift-default-5764cb76cf-9mpbm"]
Oct 30 19:33:53.540516 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.540438    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" path="/var/lib/kubelet/pods/1c908ea0-f632-4c83-b6c9-440f60a50c99/volumes"
Oct 30 19:33:53.620305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:53.620253    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"]
Oct 30 19:33:53.625329 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:33:53.625299    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bda85a4_37bb_4f92_8e42_5f44a6e7fdf0.slice/crio-f53878593ea2bbbc20ebdf6be85a0c6f6edb0b56f0392242a0aab264b054064d WatchSource:0}: Error finding container f53878593ea2bbbc20ebdf6be85a0c6f6edb0b56f0392242a0aab264b054064d: Status 404 returned error can't find the container with id f53878593ea2bbbc20ebdf6be85a0c6f6edb0b56f0392242a0aab264b054064d
Oct 30 19:33:54.129059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:54.129023    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz" event={"ID":"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0","Type":"ContainerStarted","Data":"f53878593ea2bbbc20ebdf6be85a0c6f6edb0b56f0392242a0aab264b054064d"}
Oct 30 19:33:55.138293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:55.138258    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz" event={"ID":"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0","Type":"ContainerStarted","Data":"5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861"}
Oct 30 19:33:55.138695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:55.138327    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:33:55.255397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:33:55.255344    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz" podStartSLOduration=2.338007827 podStartE2EDuration="3.255324598s" totalImagesPullingTime="917.316771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:55.252783375 +0000 UTC m=+2827.837266371" watchObservedRunningTime="2026-10-30 19:33:55.255324598 +0000 UTC m=+2827.839807589"
Oct 30 19:34:01.150176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:01.150145    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:34:11.009461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:11.009428    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"]
Oct 30 19:34:11.009968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:11.009647    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy" containerID="cri-o://f3180eb5c1cbec718626980cd6ca5959b4e3a997567b9d714d099bb44da3d1ac" gracePeriod=30
Oct 30 19:34:15.962925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:15.962873    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:34:15.963530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:15.962962    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:34:16.197286 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:34:16.197242    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:34:16.287250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.287211    5232 generic.go:356] "Generic (PLEG): container finished" podID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerID="f3180eb5c1cbec718626980cd6ca5959b4e3a997567b9d714d099bb44da3d1ac" exitCode=0
Oct 30 19:34:16.287388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.287285    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" event={"ID":"fc314899-ffa0-49eb-a97b-1bde15e33c58","Type":"ContainerDied","Data":"f3180eb5c1cbec718626980cd6ca5959b4e3a997567b9d714d099bb44da3d1ac"}
Oct 30 19:34:16.287388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.287360    5232 scope.go:122] "RemoveContainer" containerID="ededd401189e94debc6ed3d260571c9ecd4706a434c96977ad4d169b8815f2aa"
Oct 30 19:34:16.312331 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.312309    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:34:16.385940 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.385905    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-certs\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.385953    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-socket\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386092    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-token\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-token\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386161    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-envoy\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386216    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-certs" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:16.386250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386217    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-podinfo\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386308    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-socket" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:16.386397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386321    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-credential-socket\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386351    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-data\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386412    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-kube-api-access-xdxdl\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-kube-api-access-xdxdl\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386462    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc314899-ffa0-49eb-a97b-1bde15e33c58-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc314899-ffa0-49eb-a97b-1bde15e33c58-istiod-ca-cert\") pod \"fc314899-ffa0-49eb-a97b-1bde15e33c58\" (UID: \"fc314899-ffa0-49eb-a97b-1bde15e33c58\") "
Oct 30 19:34:16.386683 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386579    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-credential-socket" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:16.386934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386907    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-data" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:16.387053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.386976    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc314899-ffa0-49eb-a97b-1bde15e33c58-istiod-ca-cert" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:16.387120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.387049    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.387120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.387072    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.387120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.387088    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.387120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.387102    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.388391 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.388370    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-podinfo" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:34:16.388620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.388596    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-token" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:16.388708 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.388627    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-envoy" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:16.389952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.389920    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-kube-api-access-xdxdl" pod "fc314899-ffa0-49eb-a97b-1bde15e33c58" (UID: "fc314899-ffa0-49eb-a97b-1bde15e33c58"). InnerVolumeSpecName "kube-api-access-xdxdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:16.487786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.487753    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.487786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.487782    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdxdl\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-kube-api-access-xdxdl\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.488004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.487794    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc314899-ffa0-49eb-a97b-1bde15e33c58-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.488004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.487821    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.488004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.487834    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc314899-ffa0-49eb-a97b-1bde15e33c58-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:16.711182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.711149    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"]
Oct 30 19:34:16.711670 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:16.711636    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy" containerID="cri-o://89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150" gracePeriod=30
Oct 30 19:34:17.294552 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:17.294513    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq" event={"ID":"fc314899-ffa0-49eb-a97b-1bde15e33c58","Type":"ContainerDied","Data":"afadadb58b0cf1f0b2df4e669a37aa3d4f1da97cf942ee8cf54ca22a72905cf0"}
Oct 30 19:34:17.295032 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:17.294571    5232 scope.go:122] "RemoveContainer" containerID="f3180eb5c1cbec718626980cd6ca5959b4e3a997567b9d714d099bb44da3d1ac"
Oct 30 19:34:17.295032 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:17.294521    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"
Oct 30 19:34:17.582579 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:17.582488    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"]
Oct 30 19:34:17.591233 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:17.591183    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-f9a20d-openshift-default-f4cbdb889-vfdkq"]
Oct 30 19:34:19.557551 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:19.557493    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" path="/var/lib/kubelet/pods/fc314899-ffa0-49eb-a97b-1bde15e33c58/volumes"
Oct 30 19:34:21.987541 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:21.987478    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:34:22.154095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154053    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-kube-api-access-ch4g5\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-kube-api-access-ch4g5\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154159    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73a453db-4937-4dab-8254-c2decafd1fb8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a453db-4937-4dab-8254-c2decafd1fb8-istiod-ca-cert\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154222    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-data\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154274    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-certs\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154336    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/73a453db-4937-4dab-8254-c2decafd1fb8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a453db-4937-4dab-8254-c2decafd1fb8-istio-podinfo\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154365    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-envoy\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154455    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-istio-token\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-istio-token\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154520    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-socket\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154674 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154579    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-credential-socket\") pod \"73a453db-4937-4dab-8254-c2decafd1fb8\" (UID: \"73a453db-4937-4dab-8254-c2decafd1fb8\") "
Oct 30 19:34:22.154674 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154625    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-certs" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:22.154674 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154636    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a453db-4937-4dab-8254-c2decafd1fb8-istiod-ca-cert" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:22.154956 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154874    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-socket" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:22.154956 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.154909    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-data" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:22.155234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.155105    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-credential-socket" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:22.155234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.155118    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.155234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.155140    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.155234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.155154    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a453db-4937-4dab-8254-c2decafd1fb8-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.155234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.155167    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.158778 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.158741    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-kube-api-access-ch4g5" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "kube-api-access-ch4g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:22.158778 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.158752    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-istio-token" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:22.159026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.158854    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73a453db-4937-4dab-8254-c2decafd1fb8-istio-podinfo" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:34:22.159758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.159731    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-envoy" pod "73a453db-4937-4dab-8254-c2decafd1fb8" (UID: "73a453db-4937-4dab-8254-c2decafd1fb8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:22.256349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.256263    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.256349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.256297    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch4g5\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-kube-api-access-ch4g5\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.256349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.256308    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a453db-4937-4dab-8254-c2decafd1fb8-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.256349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.256317    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a453db-4937-4dab-8254-c2decafd1fb8-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.256349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.256325    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a453db-4937-4dab-8254-c2decafd1fb8-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:22.328882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.328846    5232 generic.go:356] "Generic (PLEG): container finished" podID="73a453db-4937-4dab-8254-c2decafd1fb8" containerID="89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150" exitCode=0
Oct 30 19:34:22.329044 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.328918    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"
Oct 30 19:34:22.329044 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.328937    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" event={"ID":"73a453db-4937-4dab-8254-c2decafd1fb8","Type":"ContainerDied","Data":"89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150"}
Oct 30 19:34:22.329044 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.329029    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96" event={"ID":"73a453db-4937-4dab-8254-c2decafd1fb8","Type":"ContainerDied","Data":"be61673f595004a02ad7fa6bff7ec890f5e458c45c12cf130cb47f0ad2c4d319"}
Oct 30 19:34:22.329166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.329053    5232 scope.go:122] "RemoveContainer" containerID="89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150"
Oct 30 19:34:22.339121 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.339099    5232 scope.go:122] "RemoveContainer" containerID="fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f"
Oct 30 19:34:22.351621 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.351592    5232 scope.go:122] "RemoveContainer" containerID="89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150"
Oct 30 19:34:22.351989 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:34:22.351966    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150\": container with ID starting with 89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150 not found: ID does not exist" containerID="89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150"
Oct 30 19:34:22.352055 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.352003    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150"} err="failed to get container status \"89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150\": rpc error: code = NotFound desc = could not find container \"89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150\": container with ID starting with 89a4e4e07a77cb41503ecbbe98ce04096c70a50578d707e185206ac5f120d150 not found: ID does not exist"
Oct 30 19:34:22.352055 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.352020    5232 scope.go:122] "RemoveContainer" containerID="fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f"
Oct 30 19:34:22.352293 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:34:22.352270    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f\": container with ID starting with fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f not found: ID does not exist" containerID="fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f"
Oct 30 19:34:22.352360 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.352303    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f"} err="failed to get container status \"fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f\": rpc error: code = NotFound desc = could not find container \"fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f\": container with ID starting with fc0e8261a45c9859beede803b91d526fda44094de32c0bcbdc2526f65ef2d78f not found: ID does not exist"
Oct 30 19:34:22.490490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.490453    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"]
Oct 30 19:34:22.501280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:22.501248    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-441836-openshift-default-5f67668d9c-vgg96"]
Oct 30 19:34:23.336879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:23.336849    5232 generic.go:356] "Generic (PLEG): container finished" podID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerID="bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13" exitCode=0
Oct 30 19:34:23.337328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:23.336919    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" event={"ID":"504ff7c3-69c5-4422-a4b2-ba43ecf1197f","Type":"ContainerDied","Data":"bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13"}
Oct 30 19:34:23.337399 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:23.337378    5232 scope.go:122] "RemoveContainer" containerID="bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13"
Oct 30 19:34:23.338942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:23.338912    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:23.339017 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:23.338981    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:23.339017 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:23.339013    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:23.542609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:23.542571    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" path="/var/lib/kubelet/pods/73a453db-4937-4dab-8254-c2decafd1fb8/volumes"
Oct 30 19:34:24.368681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:24.368648    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" event={"ID":"504ff7c3-69c5-4422-a4b2-ba43ecf1197f","Type":"ContainerStarted","Data":"9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d"}
Oct 30 19:34:24.662160 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:24.662058    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:34:24.663511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:24.663483    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:34:24.663643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:24.663549    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 19:34:25.662278 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:25.662236    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:34:25.662791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:25.662302    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 19:34:26.662316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:26.662277    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:34:26.662772 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:26.662352    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 19:34:27.572328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:27.572293    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:34:27.662179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:27.662135    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:34:27.662362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:27.662202    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 19:34:28.662257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:28.662210    5232 patch_prober.go:29] interesting pod/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:34:28.662445 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:28.662272    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 19:34:29.664744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:29.664711    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:34:29.665838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:29.665792    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:34:45.073206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.073163    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"]
Oct 30 19:34:45.073758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.073443    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh" podUID="872dc425-c1d5-45d8-99f8-e62d6e67f54a" containerName="maas-api" containerID="cri-o://2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377" gracePeriod=30
Oct 30 19:34:45.342152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.342124    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:34:45.409994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.409961    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/872dc425-c1d5-45d8-99f8-e62d6e67f54a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/872dc425-c1d5-45d8-99f8-e62d6e67f54a-openshift-service-ca\") pod \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") "
Oct 30 19:34:45.410166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.410070    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-metrics-certs\") pod \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") "
Oct 30 19:34:45.410166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.410116    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls\") pod \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") "
Oct 30 19:34:45.410259 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.410170    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/872dc425-c1d5-45d8-99f8-e62d6e67f54a-kube-api-access-694nx\" (UniqueName: \"kubernetes.io/projected/872dc425-c1d5-45d8-99f8-e62d6e67f54a-kube-api-access-694nx\") pod \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\" (UID: \"872dc425-c1d5-45d8-99f8-e62d6e67f54a\") "
Oct 30 19:34:45.410395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.410364    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/872dc425-c1d5-45d8-99f8-e62d6e67f54a-openshift-service-ca" pod "872dc425-c1d5-45d8-99f8-e62d6e67f54a" (UID: "872dc425-c1d5-45d8-99f8-e62d6e67f54a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:45.410546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.410530    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/872dc425-c1d5-45d8-99f8-e62d6e67f54a-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:45.412263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.412236    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls" pod "872dc425-c1d5-45d8-99f8-e62d6e67f54a" (UID: "872dc425-c1d5-45d8-99f8-e62d6e67f54a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:45.412263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.412249    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-metrics-certs" pod "872dc425-c1d5-45d8-99f8-e62d6e67f54a" (UID: "872dc425-c1d5-45d8-99f8-e62d6e67f54a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:45.412422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.412284    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872dc425-c1d5-45d8-99f8-e62d6e67f54a-kube-api-access-694nx" pod "872dc425-c1d5-45d8-99f8-e62d6e67f54a" (UID: "872dc425-c1d5-45d8-99f8-e62d6e67f54a"). InnerVolumeSpecName "kube-api-access-694nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:45.511907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.511874    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-694nx\" (UniqueName: \"kubernetes.io/projected/872dc425-c1d5-45d8-99f8-e62d6e67f54a-kube-api-access-694nx\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:45.511907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.511901    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:45.511907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.511913    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/872dc425-c1d5-45d8-99f8-e62d6e67f54a-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:45.529392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.529358    5232 generic.go:356] "Generic (PLEG): container finished" podID="872dc425-c1d5-45d8-99f8-e62d6e67f54a" containerID="2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377" exitCode=0
Oct 30 19:34:45.529559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.529409    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"
Oct 30 19:34:45.529559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.529439    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh" event={"ID":"872dc425-c1d5-45d8-99f8-e62d6e67f54a","Type":"ContainerDied","Data":"2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377"}
Oct 30 19:34:45.529559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.529492    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh" event={"ID":"872dc425-c1d5-45d8-99f8-e62d6e67f54a","Type":"ContainerDied","Data":"288e483ea9f4e016f2c8c10d8d17b47b0b8d66cc78ea8a536671b717d80c64fe"}
Oct 30 19:34:45.529559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.529514    5232 scope.go:122] "RemoveContainer" containerID="2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377"
Oct 30 19:34:45.540938 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.540922    5232 scope.go:122] "RemoveContainer" containerID="2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377"
Oct 30 19:34:45.541188 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:34:45.541169    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377\": container with ID starting with 2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377 not found: ID does not exist" containerID="2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377"
Oct 30 19:34:45.541252 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.541197    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377"} err="failed to get container status \"2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377\": rpc error: code = NotFound desc = could not find container \"2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377\": container with ID starting with 2e70e39af92b88d98245d53388de2c2e885e7dd99a0646bd2c634cec844be377 not found: ID does not exist"
Oct 30 19:34:45.689660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.689618    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"]
Oct 30 19:34:45.694037 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:45.694007    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-69d429-b57878656-qxsmh"]
Oct 30 19:34:46.187250 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:34:46.187213    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:34:47.540443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:47.540410    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="872dc425-c1d5-45d8-99f8-e62d6e67f54a" path="/var/lib/kubelet/pods/872dc425-c1d5-45d8-99f8-e62d6e67f54a/volumes"
Oct 30 19:34:52.131821 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.131768    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"]
Oct 30 19:34:52.132350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132282    5232 cpu_manager.go:414] "containerMap: removing container" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy"
Oct 30 19:34:52.132350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132298    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy"
Oct 30 19:34:52.132350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132329    5232 cpu_manager.go:414] "containerMap: removing container" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy"
Oct 30 19:34:52.132350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132335    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132353    5232 cpu_manager.go:414] "containerMap: removing container" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132362    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132372    5232 cpu_manager.go:414] "containerMap: removing container" podUID="872dc425-c1d5-45d8-99f8-e62d6e67f54a" containerName="maas-api"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132380    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="872dc425-c1d5-45d8-99f8-e62d6e67f54a" containerName="maas-api"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132465    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132479    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132486    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="73a453db-4937-4dab-8254-c2decafd1fb8" containerName="istio-proxy"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132492    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="872dc425-c1d5-45d8-99f8-e62d6e67f54a" containerName="maas-api"
Oct 30 19:34:52.132530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.132500    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c908ea0-f632-4c83-b6c9-440f60a50c99" containerName="istio-proxy"
Oct 30 19:34:52.137354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.137334    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.238507 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.238468    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"]
Oct 30 19:34:52.292504 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.292463    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.292681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.292558    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.292681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.292615    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.292681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.292632    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.292681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.292652    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phvtz\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-kube-api-access-phvtz\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.292902 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.292874    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.292945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.292932    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.293006 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.292992    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.293097 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.293072    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394006 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.393923    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394006 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.393964    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394019    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394037    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394062    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phvtz\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-kube-api-access-phvtz\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394112    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394147    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394202    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394238    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394445    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394526    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394669    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.394759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394678    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.395017 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.394994    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.396353 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.396333    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.396477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.396457    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.458430 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.458393    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.459331 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.459305    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phvtz\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-kube-api-access-phvtz\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:52.749953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:52.749869    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:53.180693 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:53.180655    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"]
Oct 30 19:34:53.185210 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:34:53.185185    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18f59f9_7bd1_4691_b69e_9ab4e4134f5d.slice/crio-5b0fadeaede4aed2d1a6d550201809140db0ee6faef2f27c6cab4ac98233d766 WatchSource:0}: Error finding container 5b0fadeaede4aed2d1a6d550201809140db0ee6faef2f27c6cab4ac98233d766: Status 404 returned error can't find the container with id 5b0fadeaede4aed2d1a6d550201809140db0ee6faef2f27c6cab4ac98233d766
Oct 30 19:34:53.187619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:53.187584    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:53.187703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:53.187650    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:53.187703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:53.187681    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:53.587882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:53.587846    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" event={"ID":"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d","Type":"ContainerStarted","Data":"62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5"}
Oct 30 19:34:53.588049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:53.587912    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" event={"ID":"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d","Type":"ContainerStarted","Data":"5b0fadeaede4aed2d1a6d550201809140db0ee6faef2f27c6cab4ac98233d766"}
Oct 30 19:34:53.750087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:53.750051    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:34:53.813957 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:53.813903    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" podStartSLOduration=1.813889844 podStartE2EDuration="1.813889844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:53.765631722 +0000 UTC m=+2886.350114721" watchObservedRunningTime="2026-10-30 19:34:53.813889844 +0000 UTC m=+2886.398372876"
Oct 30 19:34:54.776843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:54.771315    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:34:54.776843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:54.771380    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 19:34:55.758318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:55.758268    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:34:55.758614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:55.758583    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 19:34:56.759897 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:56.759166    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:34:56.759897 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:56.759230    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 19:34:57.765835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:57.758165    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:34:57.765835 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:57.758250    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 19:34:58.754836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:58.751123    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:34:58.754836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:58.751199    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 19:34:59.751142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:59.751103    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:34:59.751547 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:34:59.751158    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 19:35:00.804627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:00.804590    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:35:00.805412 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:00.805389    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:35:00.806631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:00.806612    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:35:05.360516 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.360480    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"]
Oct 30 19:35:05.361330 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.360733    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs" podUID="8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" containerName="maas-api" containerID="cri-o://f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a" gracePeriod=30
Oct 30 19:35:05.620949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.620860    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:35:05.664630 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.664587    5232 generic.go:356] "Generic (PLEG): container finished" podID="8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" containerID="f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a" exitCode=0
Oct 30 19:35:05.664806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.664637    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"
Oct 30 19:35:05.664806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.664674    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs" event={"ID":"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f","Type":"ContainerDied","Data":"f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a"}
Oct 30 19:35:05.664806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.664736    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs" event={"ID":"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f","Type":"ContainerDied","Data":"29e331215c302fdf259dc8d0e7160328dce3c4b15df817c0834cda49e7f3e570"}
Oct 30 19:35:05.664806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.664762    5232 scope.go:122] "RemoveContainer" containerID="f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a"
Oct 30 19:35:05.671225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.671191    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-metrics-certs\") pod \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") "
Oct 30 19:35:05.671380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.671254    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-openshift-service-ca\") pod \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") "
Oct 30 19:35:05.671380 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.671349    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls\") pod \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") "
Oct 30 19:35:05.671470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.671437    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-kube-api-access-9q9mp\" (UniqueName: \"kubernetes.io/projected/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-kube-api-access-9q9mp\") pod \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\" (UID: \"8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f\") "
Oct 30 19:35:05.671686 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.671656    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-openshift-service-ca" pod "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" (UID: "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:05.671999 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.671978    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:05.673435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.673406    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-metrics-certs" pod "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" (UID: "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:05.673666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.673646    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-kube-api-access-9q9mp" pod "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" (UID: "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f"). InnerVolumeSpecName "kube-api-access-9q9mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:05.673731 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.673709    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls" pod "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" (UID: "8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:05.675531 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.675510    5232 scope.go:122] "RemoveContainer" containerID="f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a"
Oct 30 19:35:05.675883 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:35:05.675859    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a\": container with ID starting with f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a not found: ID does not exist" containerID="f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a"
Oct 30 19:35:05.675952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.675897    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a"} err="failed to get container status \"f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a\": rpc error: code = NotFound desc = could not find container \"f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a\": container with ID starting with f4f710a2f3358b2ef9d60cd2704771988a8813bef20e09ccfae7bb9fccb19d9a not found: ID does not exist"
Oct 30 19:35:05.773739 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.773703    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:05.773739 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.773734    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:05.773739 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:05.773745    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q9mp\" (UniqueName: \"kubernetes.io/projected/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f-kube-api-access-9q9mp\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:06.084111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:06.084073    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"]
Oct 30 19:35:06.088163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:06.088133    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-e73f1a-5dbfb65c87-vvmzs"]
Oct 30 19:35:07.541398 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:07.541361    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" path="/var/lib/kubelet/pods/8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f/volumes"
Oct 30 19:35:16.187399 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:35:16.187363    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:35:46.187458 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:35:46.187420    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:35:52.105130 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.105095    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"]
Oct 30 19:35:52.105671 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.105652    5232 cpu_manager.go:414] "containerMap: removing container" podUID="8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" containerName="maas-api"
Oct 30 19:35:52.105724 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.105672    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" containerName="maas-api"
Oct 30 19:35:52.105724 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.105700    5232 cpu_manager.go:414] "containerMap: removing container" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy"
Oct 30 19:35:52.105724 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.105705    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy"
Oct 30 19:35:52.105844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.105788    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc314899-ffa0-49eb-a97b-1bde15e33c58" containerName="istio-proxy"
Oct 30 19:35:52.105844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.105802    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="8eab9c9e-4864-4f6e-89eb-b0d4ee4ff12f" containerName="maas-api"
Oct 30 19:35:52.110109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.110091    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.175898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.175863    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.175898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.175901    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xglzz\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-kube-api-access-xglzz\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.176197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.175991    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.176197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.176046    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.176197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.176110    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.176197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.176179    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.176390 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.176215    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.176390 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.176253    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.176477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.176386    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.212667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.212632    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"]
Oct 30 19:35:52.278059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278022    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278065    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglzz\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-kube-api-access-xglzz\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278106    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278140    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278207    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278243    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278294    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278651 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278626    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278688    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.278941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278915    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.279063 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.279016    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.279132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.278362    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.279194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.279163    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.279492 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.279466    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.280636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.280608    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.280786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.280764    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.339607 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.339575    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.339936 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.339909    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xglzz\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-kube-api-access-xglzz\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.423195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.423110    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:52.853683 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.853545    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"]
Oct 30 19:35:52.857964 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:35:52.857925    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c9cad5_cd7c_4c0d_a6c4_9fd55e221ed7.slice/crio-3e313d6513a928027b8b408b24ac3939f4c9407eece5b58a2908f1108cf78dba WatchSource:0}: Error finding container 3e313d6513a928027b8b408b24ac3939f4c9407eece5b58a2908f1108cf78dba: Status 404 returned error can't find the container with id 3e313d6513a928027b8b408b24ac3939f4c9407eece5b58a2908f1108cf78dba
Oct 30 19:35:52.860388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.860353    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:52.860466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.860418    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:52.860466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.860451    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:52.975504 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.975461    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" event={"ID":"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7","Type":"ContainerStarted","Data":"ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92"}
Oct 30 19:35:52.975717 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:52.975531    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" event={"ID":"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7","Type":"ContainerStarted","Data":"3e313d6513a928027b8b408b24ac3939f4c9407eece5b58a2908f1108cf78dba"}
Oct 30 19:35:53.095084 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:53.095028    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podStartSLOduration=1.095013035 podStartE2EDuration="1.095013035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:53.093140131 +0000 UTC m=+2945.677623127" watchObservedRunningTime="2026-10-30 19:35:53.095013035 +0000 UTC m=+2945.679496131"
Oct 30 19:35:53.424261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:53.424225    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:35:53.425872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:53.425845    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:35:53.425974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:53.425902    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 19:35:54.423537 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:54.423500    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:35:54.423720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:54.423568    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 19:35:55.424490 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:55.424450    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:35:55.424916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:55.424509    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 19:35:56.424515 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.424478    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:35:56.425066 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.424543    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 19:35:56.482746 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.482702    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"]
Oct 30 19:35:56.489257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.489220    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.561971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.546790    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-9275c3-openshift-default-dockercfg-hq6vr\""
Oct 30 19:35:56.598310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.598273    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"]
Oct 30 19:35:56.633126 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633087    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.633327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633130    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.633327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633160    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.633327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633254    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.633327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633296    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.633327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633315    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpqjd\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-kube-api-access-fpqjd\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.633631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633406    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.633631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633454    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.633631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.633562    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.734915    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.734970    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735026    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735046    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735072    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735129    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735174    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735202    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpqjd\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-kube-api-access-fpqjd\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735245    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735530    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735654    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735697    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.735971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.735808    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.736321 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.736301    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.768234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.768194    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.768450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.768420    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.794739 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.794709    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.795066 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.795043    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpqjd\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-kube-api-access-fpqjd\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:56.808596 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:56.808567    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:57.264195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:57.264147    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"]
Oct 30 19:35:57.264350 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:35:57.264147    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b2b627_fd1a_4e81_8ee3_de291fb99da2.slice/crio-682e3c4982be4dc9d7bd47a372696ac972bb789a1217ea478ff80ab438433ecb WatchSource:0}: Error finding container 682e3c4982be4dc9d7bd47a372696ac972bb789a1217ea478ff80ab438433ecb: Status 404 returned error can't find the container with id 682e3c4982be4dc9d7bd47a372696ac972bb789a1217ea478ff80ab438433ecb
Oct 30 19:35:57.266912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:57.266875    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:57.267005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:57.266949    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:57.267054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:57.267007    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:57.424157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:57.424122    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:35:57.424328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:57.424180    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 19:35:58.010408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:58.010376    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" event={"ID":"16b2b627-fd1a-4e81-8ee3-de291fb99da2","Type":"ContainerStarted","Data":"f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429"}
Oct 30 19:35:58.010887 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:58.010864    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" event={"ID":"16b2b627-fd1a-4e81-8ee3-de291fb99da2","Type":"ContainerStarted","Data":"682e3c4982be4dc9d7bd47a372696ac972bb789a1217ea478ff80ab438433ecb"}
Oct 30 19:35:58.140840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:58.137264    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podStartSLOduration=2.137246459 podStartE2EDuration="2.137246459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:58.135205289 +0000 UTC m=+2950.719688287" watchObservedRunningTime="2026-10-30 19:35:58.137246459 +0000 UTC m=+2950.721729460"
Oct 30 19:35:58.424220 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:58.424184    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:35:58.424395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:58.424243    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 19:35:58.809722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:58.809630    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:35:58.811068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:58.811037    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:35:58.811194 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:58.811092    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 19:35:59.424114 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:59.424076    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:35:59.424515 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:59.424132    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 30 19:35:59.808988 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:59.808893    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:35:59.808988 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:35:59.808946    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 19:36:00.427680 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:00.427646    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:36:00.428193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:00.428066    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:36:00.428698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:00.428679    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:36:00.809095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:00.808998    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:36:00.809095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:00.809055    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 19:36:01.808981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:01.808922    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:36:01.809464 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:01.808991    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 19:36:02.809838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:02.809775    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:36:02.810269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:02.809871    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 19:36:03.809698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:03.809657    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:36:03.809908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:03.809720    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 19:36:04.110169 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:04.110143    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-9zk2f_528babed-84aa-491f-85a9-16cc40b8edc9/payload-processing/0.log"
Oct 30 19:36:04.366738 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:04.366639    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-9zk2f_528babed-84aa-491f-85a9-16cc40b8edc9/payload-processing/0.log"
Oct 30 19:36:05.048215 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:05.048155    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:36:05.071219 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:05.067614    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:36:05.092046 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:05.092012    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:36:16.187660 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:36:16.187622    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:36:22.123285 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.123242    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"]
Oct 30 19:36:22.127199 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.127180    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.232275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.232241    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"]
Oct 30 19:36:22.294970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.294927    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/691fea7b-46d8-4d38-878b-14623681a293-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.295178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.294977    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fh4q\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-kube-api-access-6fh4q\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.295178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.294999    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.295178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.295056    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.295178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.295159    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.295365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.295225    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.295365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.295282    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/691fea7b-46d8-4d38-878b-14623681a293-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.295445 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.295409    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.295480 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.295452    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.397534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397418    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.397534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397507    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.397787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397574    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/691fea7b-46d8-4d38-878b-14623681a293-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.397787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397611    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fh4q\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-kube-api-access-6fh4q\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.397787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397641    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.397787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397669    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.397787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397706    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.397787 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397744    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.398242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.397807    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/691fea7b-46d8-4d38-878b-14623681a293-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.398242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.398086    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.398471 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.398434    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.398880 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.398803    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/691fea7b-46d8-4d38-878b-14623681a293-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.404714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.404664    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/691fea7b-46d8-4d38-878b-14623681a293-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.408634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.407803    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.408634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.408230    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.410362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.410339    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.462073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.462037    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.465176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.465148    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fh4q\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-kube-api-access-6fh4q\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:22.741521 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:22.741428    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:23.178127 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:23.178067    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"]
Oct 30 19:36:23.182880 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:36:23.182851    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691fea7b_46d8_4d38_878b_14623681a293.slice/crio-335996374f7ae5fbe6053d64940c552c9d481916a9a6e509e44587c4c0cdaf0a WatchSource:0}: Error finding container 335996374f7ae5fbe6053d64940c552c9d481916a9a6e509e44587c4c0cdaf0a: Status 404 returned error can't find the container with id 335996374f7ae5fbe6053d64940c552c9d481916a9a6e509e44587c4c0cdaf0a
Oct 30 19:36:23.185351 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:23.185318    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:23.185427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:23.185390    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:23.185427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:23.185419    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:23.266424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:23.266387    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" event={"ID":"691fea7b-46d8-4d38-878b-14623681a293","Type":"ContainerStarted","Data":"2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4"}
Oct 30 19:36:23.266623 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:23.266448    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" event={"ID":"691fea7b-46d8-4d38-878b-14623681a293","Type":"ContainerStarted","Data":"335996374f7ae5fbe6053d64940c552c9d481916a9a6e509e44587c4c0cdaf0a"}
Oct 30 19:36:23.385409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:23.385357    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" podStartSLOduration=1.385342901 podStartE2EDuration="1.385342901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:23.384091837 +0000 UTC m=+2975.968574846" watchObservedRunningTime="2026-10-30 19:36:23.385342901 +0000 UTC m=+2975.969825964"
Oct 30 19:36:23.760837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:23.751928    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:24.748013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:24.747977    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:36:24.748615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:24.748581    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:36:25.758840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:25.747206    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:36:25.758840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:25.747272    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:36:26.762891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:26.762002    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:36:26.762891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:26.762117    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:36:27.757130 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:27.756672    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:36:27.757130 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:27.756736    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:36:28.759215 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:28.759179    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:36:28.759706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:28.759248    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:36:29.742195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:29.742158    5232 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:36:29.742425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:29.742214    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:36:30.746199 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:30.746162    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:30.746581 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:30.746553    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:30.747221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:30.747198    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:36:41.485033 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:41.484999    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"]
Oct 30 19:36:41.485466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:41.485251    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy" containerID="cri-o://a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2" gracePeriod=30
Oct 30 19:36:45.824566 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:45.824531    5232 patch_prober.go:29] interesting pod/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:36:45.824980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:45.824592    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:36:46.187971 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:36:46.187935    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:36:46.746580 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.746520    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:36:46.762110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762081    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-data\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762173    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-credential-socket\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762243    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-kube-api-access-9g9lj\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-kube-api-access-9g9lj\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762290    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-envoy\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762329    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/43d1871f-405c-4db4-a424-4662b1e910cf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d1871f-405c-4db4-a424-4662b1e910cf-istio-podinfo\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762360    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-certs\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762381    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-data" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:36:46.762553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762417    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-socket\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762435    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-credential-socket" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:36:46.762553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762484    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-istio-token\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-istio-token\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762535    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43d1871f-405c-4db4-a424-4662b1e910cf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d1871f-405c-4db4-a424-4662b1e910cf-istiod-ca-cert\") pod \"43d1871f-405c-4db4-a424-4662b1e910cf\" (UID: \"43d1871f-405c-4db4-a424-4662b1e910cf\") "
Oct 30 19:36:46.762849 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762624    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-certs" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:36:46.762849 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.762751    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-socket" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:36:46.763088 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.763055    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d1871f-405c-4db4-a424-4662b1e910cf-istiod-ca-cert" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:46.763088 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.763081    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:46.763225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.763102    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:46.763225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.763116    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:46.763225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.763130    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:46.764580 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.764555    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-envoy" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:36:46.764972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.764947    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-kube-api-access-9g9lj" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "kube-api-access-9g9lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:46.765084 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.765063    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-istio-token" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:46.765148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.765100    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43d1871f-405c-4db4-a424-4662b1e910cf-istio-podinfo" pod "43d1871f-405c-4db4-a424-4662b1e910cf" (UID: "43d1871f-405c-4db4-a424-4662b1e910cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:36:46.864649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.864610    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:46.864649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.864642    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d1871f-405c-4db4-a424-4662b1e910cf-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:46.864649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.864652    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g9lj\" (UniqueName: \"kubernetes.io/projected/43d1871f-405c-4db4-a424-4662b1e910cf-kube-api-access-9g9lj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:46.865280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.864663    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d1871f-405c-4db4-a424-4662b1e910cf-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:46.865280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:46.864671    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d1871f-405c-4db4-a424-4662b1e910cf-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:47.467389 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.467345    5232 generic.go:356] "Generic (PLEG): container finished" podID="43d1871f-405c-4db4-a424-4662b1e910cf" containerID="a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2" exitCode=0
Oct 30 19:36:47.468132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.467450    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" event={"ID":"43d1871f-405c-4db4-a424-4662b1e910cf","Type":"ContainerDied","Data":"a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2"}
Oct 30 19:36:47.468132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.467480    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"
Oct 30 19:36:47.468132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.467545    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9" event={"ID":"43d1871f-405c-4db4-a424-4662b1e910cf","Type":"ContainerDied","Data":"162a0c1adc35d48692897adea961036f463c071b117dc7e9daa7eeb72bd89dd7"}
Oct 30 19:36:47.468132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.467595    5232 scope.go:122] "RemoveContainer" containerID="a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2"
Oct 30 19:36:47.479314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.479292    5232 scope.go:122] "RemoveContainer" containerID="fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2"
Oct 30 19:36:47.488649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.488629    5232 scope.go:122] "RemoveContainer" containerID="a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2"
Oct 30 19:36:47.488901 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:36:47.488878    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2\": container with ID starting with a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2 not found: ID does not exist" containerID="a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2"
Oct 30 19:36:47.488979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.488909    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2"} err="failed to get container status \"a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2\": rpc error: code = NotFound desc = could not find container \"a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2\": container with ID starting with a27b9f6a14e63202e544cfa9f24ece0a28258603eeac76e5e0dab6f56e285eb2 not found: ID does not exist"
Oct 30 19:36:47.488979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.488928    5232 scope.go:122] "RemoveContainer" containerID="fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2"
Oct 30 19:36:47.489184 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:36:47.489165    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2\": container with ID starting with fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2 not found: ID does not exist" containerID="fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2"
Oct 30 19:36:47.489234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.489190    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2"} err="failed to get container status \"fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2\": rpc error: code = NotFound desc = could not find container \"fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2\": container with ID starting with fea35163c177922eaade9efc0cdc2cd1590bae089097702499a77dffc094dfc2 not found: ID does not exist"
Oct 30 19:36:47.676908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.676855    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"]
Oct 30 19:36:47.682613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.682585    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-f3b5af-openshift-default-7bc78dcc69-qljg9"]
Oct 30 19:36:47.984314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.984289    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:36:47.990284 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:47.990257    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:36:48.015402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:48.015377    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:36:48.019913 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:48.019890    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:36:49.540878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:49.540838    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" path="/var/lib/kubelet/pods/43d1871f-405c-4db4-a424-4662b1e910cf/volumes"
Oct 30 19:36:57.507793 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.507749    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"]
Oct 30 19:36:57.508547 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.508525    5232 cpu_manager.go:414] "containerMap: removing container" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy"
Oct 30 19:36:57.508642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.508548    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy"
Oct 30 19:36:57.508642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.508604    5232 cpu_manager.go:414] "containerMap: removing container" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy"
Oct 30 19:36:57.508642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.508614    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy"
Oct 30 19:36:57.508799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.508758    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy"
Oct 30 19:36:57.508799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.508780    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="43d1871f-405c-4db4-a424-4662b1e910cf" containerName="istio-proxy"
Oct 30 19:36:57.513552 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.513531    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599227 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599183    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599237    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599309    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599426    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8375f142-2244-4664-9dc2-f9ac7a1dd151-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599463    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gcgj\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-kube-api-access-8gcgj\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599489    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599575    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599596    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.599647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.599612    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.619834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.619780    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"]
Oct 30 19:36:57.700858 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.700795    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.700880    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.700912    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.700981    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8375f142-2244-4664-9dc2-f9ac7a1dd151-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701020    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gcgj\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-kube-api-access-8gcgj\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701049    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701156    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701189    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701204    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701421    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701575 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701551    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701637    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701662    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.701830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.701792    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8375f142-2244-4664-9dc2-f9ac7a1dd151-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.703851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.703788    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.704069 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.704049    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.760339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.760266    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.760339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.760324    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gcgj\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-kube-api-access-8gcgj\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:57.827087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:57.827054    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:58.254325 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:58.254291    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"]
Oct 30 19:36:58.258458 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:36:58.258430    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8375f142_2244_4664_9dc2_f9ac7a1dd151.slice/crio-86a286fa6b8c0ccba6425474a782e6e9badc0795e8da2d814434dad70678cd82 WatchSource:0}: Error finding container 86a286fa6b8c0ccba6425474a782e6e9badc0795e8da2d814434dad70678cd82: Status 404 returned error can't find the container with id 86a286fa6b8c0ccba6425474a782e6e9badc0795e8da2d814434dad70678cd82
Oct 30 19:36:58.261250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:58.261214    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:58.261340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:58.261283    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:58.261340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:58.261314    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:58.547106 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:58.547010    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" event={"ID":"8375f142-2244-4664-9dc2-f9ac7a1dd151","Type":"ContainerStarted","Data":"d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788"}
Oct 30 19:36:58.547106 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:58.547068    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" event={"ID":"8375f142-2244-4664-9dc2-f9ac7a1dd151","Type":"ContainerStarted","Data":"86a286fa6b8c0ccba6425474a782e6e9badc0795e8da2d814434dad70678cd82"}
Oct 30 19:36:58.667974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:58.667914    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" podStartSLOduration=1.667899077 podStartE2EDuration="1.667899077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:58.667329948 +0000 UTC m=+3011.251812948" watchObservedRunningTime="2026-10-30 19:36:58.667899077 +0000 UTC m=+3011.252382073"
Oct 30 19:36:58.827427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:58.827333    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:36:59.829506 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:59.829469    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:36:59.829952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:36:59.829535    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 19:37:00.828349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:00.828314    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:37:00.828526 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:00.828377    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 19:37:01.831124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:01.831080    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:37:01.831537 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:01.831142    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 19:37:02.828333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:02.828292    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:37:02.828527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:02.828351    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 19:37:03.828468 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:03.828428    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:37:03.828918 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:03.828495    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 19:37:04.827929 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:04.827888    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:37:04.828110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:04.827954    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 19:37:05.832803 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:05.832772    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:37:05.833279 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:05.833049    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:37:05.833720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:05.833704    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:37:12.599254 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.599216    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"]
Oct 30 19:37:12.603790 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.603768    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.654431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.654398    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-22fb18-openshift-default-dockercfg-pqpjp\""
Oct 30 19:37:12.707647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.707612    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"]
Oct 30 19:37:12.714039 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714006    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.714211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714061    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.714211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714121    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ld9s\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-kube-api-access-2ld9s\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.714211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714168    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.714211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714193    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.714437 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714228    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.714437 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714322    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.714437 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714364    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.714594 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.714429    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815325 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815291    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815371    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815410    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815436    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815483    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815515    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815822 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815559    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ld9s\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-kube-api-access-2ld9s\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815822 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815597    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815822 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815622    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.815994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.815933    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.816206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.816181    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.816277 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.816228    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.816339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.816279    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.816402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.816357    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.817832 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.817794    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.818122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.818106    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.875149 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.875077    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.875317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.875290    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ld9s\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-kube-api-access-2ld9s\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:12.917185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:12.917150    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:13.346970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:13.346934    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"]
Oct 30 19:37:13.351057 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:37:13.351029    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dfd836a_e7a7_4d9a_9cc3_a0e7f545bd7a.slice/crio-dd91258235361920469ad257c43f177dd517905c709d33559ac112eef929d9a2 WatchSource:0}: Error finding container dd91258235361920469ad257c43f177dd517905c709d33559ac112eef929d9a2: Status 404 returned error can't find the container with id dd91258235361920469ad257c43f177dd517905c709d33559ac112eef929d9a2
Oct 30 19:37:13.353449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:13.353414    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:13.353549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:13.353494    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:13.353549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:13.353539    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:13.651430 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:13.651394    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" event={"ID":"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a","Type":"ContainerStarted","Data":"301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1"}
Oct 30 19:37:13.651892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:13.651458    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" event={"ID":"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a","Type":"ContainerStarted","Data":"dd91258235361920469ad257c43f177dd517905c709d33559ac112eef929d9a2"}
Oct 30 19:37:13.779023 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:13.778940    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podStartSLOduration=1.778918659 podStartE2EDuration="1.778918659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:13.773375593 +0000 UTC m=+3026.357858590" watchObservedRunningTime="2026-10-30 19:37:13.778918659 +0000 UTC m=+3026.363401722"
Oct 30 19:37:13.918337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:13.918244    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:14.919577 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:14.919516    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:37:14.920193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:14.919630    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:37:15.918379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:15.918340    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:37:15.918710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:15.918670    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:37:16.310215 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:16.310115    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:37:16.917855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:16.917790    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:37:16.918055 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:16.917883    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:37:17.918167 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:17.918128    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:37:17.918587 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:17.918198    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:37:18.713854 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.713807    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"]
Oct 30 19:37:18.717778 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.717753    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.777734 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.777705    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-2186d8-openshift-default-dockercfg-l6s2l\""
Oct 30 19:37:18.796339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796302    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-credential-socket\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.796510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796436    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-data\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.796510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796470    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-certs\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.796510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796496    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-envoy\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.796684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796663    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46p9v\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-kube-api-access-46p9v\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.796743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796709    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-token\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.796786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796753    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istiod-ca-cert\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.796929 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796915    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-socket\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.796973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.796966    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-podinfo\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.829529 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.829467    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"]
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.900528    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-credential-socket\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.900644    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-data\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.900674    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-certs\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.900709    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-envoy\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.900776    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46p9v\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-kube-api-access-46p9v\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.900827    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-token\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.900871    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istiod-ca-cert\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.900932    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-socket\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.913840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.901007    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-podinfo\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.922132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.922076    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-data\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.923675 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.923639    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-credential-socket\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.925059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.925033    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istiod-ca-cert\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.925513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.925481    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-certs\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.927209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.927181    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:37:18.927309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.927244    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:37:18.927724 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.927699    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-socket\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.932773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.932745    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-envoy\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.964937 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.954515    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-podinfo\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.985876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.985800    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-token\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:18.987796 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:18.987762    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46p9v\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-kube-api-access-46p9v\") pod \"e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") " pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:19.034647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.034616    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:19.551188 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.551149    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"]
Oct 30 19:37:19.556797 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:37:19.556769    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78cb9d9c_5179_4dd3_a31f_a71b9bff4f38.slice/crio-200d1e7bed50ca79f912fd2ccdcf73ad467a04759e344be5a8bdf19fa3fe9f2b WatchSource:0}: Error finding container 200d1e7bed50ca79f912fd2ccdcf73ad467a04759e344be5a8bdf19fa3fe9f2b: Status 404 returned error can't find the container with id 200d1e7bed50ca79f912fd2ccdcf73ad467a04759e344be5a8bdf19fa3fe9f2b
Oct 30 19:37:19.559344 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.559305    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:19.559453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.559391    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:19.559453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.559437    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:19.726761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.726721    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" event={"ID":"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38","Type":"ContainerStarted","Data":"6224ebe6a7c20904a7e85aa0e355c9636de85ea3afc8a60498135accb5453b65"}
Oct 30 19:37:19.727000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.726786    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" event={"ID":"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38","Type":"ContainerStarted","Data":"200d1e7bed50ca79f912fd2ccdcf73ad467a04759e344be5a8bdf19fa3fe9f2b"}
Oct 30 19:37:19.845971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.845904    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" podStartSLOduration=1.8458845 podStartE2EDuration="1.8458845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:19.843805485 +0000 UTC m=+3032.428288509" watchObservedRunningTime="2026-10-30 19:37:19.8458845 +0000 UTC m=+3032.430367511"
Oct 30 19:37:19.918038 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.918002    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:37:19.918211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:19.918069    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:37:20.035164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:20.035126    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:20.037095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:20.037062    5232 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:37:20.037241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:20.037123    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 19:37:21.040772 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:21.040733    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:21.744848 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:21.742879    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:21.744848 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:21.743720    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:37:21.959455 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:21.958966    5232 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:21.959455 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:21.959036    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:22.078364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:22.078278    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:22.078808 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:22.078637    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:22.079391 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:22.079372    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:37:22.855301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:22.855267    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"]
Oct 30 19:37:22.855511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:22.855493    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz" podUID="0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" containerName="maas-api" containerID="cri-o://5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861" gracePeriod=30
Oct 30 19:37:23.132566 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.132538    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:37:23.163246 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.163214    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-metrics-certs\") pod \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") "
Oct 30 19:37:23.163409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.163281    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-maas-api-tls\") pod \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") "
Oct 30 19:37:23.163409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.163383    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-openshift-service-ca\") pod \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") "
Oct 30 19:37:23.163555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.163538    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-kube-api-access-pzh52\" (UniqueName: \"kubernetes.io/projected/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-kube-api-access-pzh52\") pod \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\" (UID: \"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0\") "
Oct 30 19:37:23.163760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.163732    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-openshift-service-ca" pod "0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" (UID: "0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:23.164002 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.163980    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:23.165491 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.165457    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-metrics-certs" pod "0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" (UID: "0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:23.165838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.165786    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-maas-api-tls" pod "0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" (UID: "0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:23.165962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.165833    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-kube-api-access-pzh52" pod "0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" (UID: "0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0"). InnerVolumeSpecName "kube-api-access-pzh52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:23.338997 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.338959    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzh52\" (UniqueName: \"kubernetes.io/projected/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-kube-api-access-pzh52\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:23.338997 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.339000    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:23.339215 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.339016    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:23.757029 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.756996    5232 generic.go:356] "Generic (PLEG): container finished" podID="0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" containerID="5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861" exitCode=0
Oct 30 19:37:23.757029 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.757039    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"
Oct 30 19:37:23.757298 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.757074    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz" event={"ID":"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0","Type":"ContainerDied","Data":"5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861"}
Oct 30 19:37:23.757298 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.757139    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz" event={"ID":"0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0","Type":"ContainerDied","Data":"f53878593ea2bbbc20ebdf6be85a0c6f6edb0b56f0392242a0aab264b054064d"}
Oct 30 19:37:23.757298 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.757159    5232 scope.go:122] "RemoveContainer" containerID="5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861"
Oct 30 19:37:23.768908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.768879    5232 scope.go:122] "RemoveContainer" containerID="5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861"
Oct 30 19:37:23.769203 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:23.769183    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861\": container with ID starting with 5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861 not found: ID does not exist" containerID="5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861"
Oct 30 19:37:23.769290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.769213    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861"} err="failed to get container status \"5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861\": rpc error: code = NotFound desc = could not find container \"5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861\": container with ID starting with 5cb585641d17f18c5a608f5c088e49ffbcf8f773c8dab992465d80f1f3e77861 not found: ID does not exist"
Oct 30 19:37:23.937082 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.935981    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"]
Oct 30 19:37:23.948339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:23.948297    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-83ac32-769dc6fc64-hfgfz"]
Oct 30 19:37:25.570837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:25.567661    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" path="/var/lib/kubelet/pods/0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0/volumes"
Oct 30 19:37:27.205559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.205521    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"]
Oct 30 19:37:27.206395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.206373    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" containerName="maas-api"
Oct 30 19:37:27.206474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.206400    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" containerName="maas-api"
Oct 30 19:37:27.206570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.206559    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bda85a4-37bb-4f92-8e42-5f44a6e7fdf0" containerName="maas-api"
Oct 30 19:37:27.211343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.211324    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299291 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299255    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299446 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299425    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299486 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299473    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299528 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299506    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299635 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299619    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299683 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299674    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299747 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299732    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299899    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.299985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.299967    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpqp7\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-kube-api-access-qpqp7\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.312101 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.312063    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"]
Oct 30 19:37:27.401093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401060    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpqp7\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-kube-api-access-qpqp7\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401154    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401189    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401213    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401234    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401262    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401315    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401347    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401570 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401403    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401674    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401700    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.401801 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401748    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.402029 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.401999    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.402151 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.402039    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.403739 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.403710    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.403940 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.403920    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.460000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.459936    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.460171 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.460149    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpqp7\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-kube-api-access-qpqp7\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.526335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.526295    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:27.965308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.965273    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"]
Oct 30 19:37:27.969038 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:37:27.969014    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf9c17c_6298_4bc1_a98f_4d4c6e67ced3.slice/crio-9f9906f9352fc5058fea296856fec7fd216352dcff7104a363de26f3216dff6b WatchSource:0}: Error finding container 9f9906f9352fc5058fea296856fec7fd216352dcff7104a363de26f3216dff6b: Status 404 returned error can't find the container with id 9f9906f9352fc5058fea296856fec7fd216352dcff7104a363de26f3216dff6b
Oct 30 19:37:27.971498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.971466    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:27.971564 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.971536    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:27.971609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:27.971567    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:28.800107 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:28.800065    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" event={"ID":"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3","Type":"ContainerStarted","Data":"654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938"}
Oct 30 19:37:28.800588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:28.800133    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" event={"ID":"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3","Type":"ContainerStarted","Data":"9f9906f9352fc5058fea296856fec7fd216352dcff7104a363de26f3216dff6b"}
Oct 30 19:37:28.968257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:28.968206    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" podStartSLOduration=1.9681912339999998 podStartE2EDuration="1.968191234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:28.96773205 +0000 UTC m=+3041.552215049" watchObservedRunningTime="2026-10-30 19:37:28.968191234 +0000 UTC m=+3041.552674231"
Oct 30 19:37:29.527357 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.527323    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:29.528634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.528611    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 19:37:29.528705 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.528672    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 19:37:29.865153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.865115    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"]
Oct 30 19:37:29.869316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.869293    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:29.919708 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.919676    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-db06cf-serving-cert\""
Oct 30 19:37:29.934457 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.934425    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:29.934641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.934614    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:29.934720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.934695    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/525f236d-76e8-4404-9c8c-5af6416906de-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:29.934991 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.934961    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn8nr\" (UniqueName: \"kubernetes.io/projected/525f236d-76e8-4404-9c8c-5af6416906de-kube-api-access-tn8nr\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:29.970301 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:29.970256    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"]
Oct 30 19:37:30.036157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.036123    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.036328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.036180    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/525f236d-76e8-4404-9c8c-5af6416906de-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.036328 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:30.036258    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-serving-cert: secret "maas-api-e2e-shared-a-w3-db06cf-serving-cert" not found
Oct 30 19:37:30.036328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.036273    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn8nr\" (UniqueName: \"kubernetes.io/projected/525f236d-76e8-4404-9c8c-5af6416906de-kube-api-access-tn8nr\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.036328 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:30.036323    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls podName:525f236d-76e8-4404-9c8c-5af6416906de nodeName:}" failed. No retries permitted until 2026-10-30 19:37:30.536307696 +0000 UTC m=+3043.120790672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls") pod "maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc" (UID: "525f236d-76e8-4404-9c8c-5af6416906de") : secret "maas-api-e2e-shared-a-w3-db06cf-serving-cert" not found
Oct 30 19:37:30.036496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.036372    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.060836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.036948    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/525f236d-76e8-4404-9c8c-5af6416906de-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.060836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.059435    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.120042 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.119945    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn8nr\" (UniqueName: \"kubernetes.io/projected/525f236d-76e8-4404-9c8c-5af6416906de-kube-api-access-tn8nr\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.527432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.527349    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 19:37:30.527571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.527428    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 19:37:30.560836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.560715    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.564722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.564695    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:30.782865 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:30.782748    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:31.183469 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:31.183358    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"]
Oct 30 19:37:31.187506 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:37:31.187481    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525f236d_76e8_4404_9c8c_5af6416906de.slice/crio-3d2721c5f3ad01a8ac7ffc57a39df1872ace2b3a7eb2c82fb3aa0317e70dc710 WatchSource:0}: Error finding container 3d2721c5f3ad01a8ac7ffc57a39df1872ace2b3a7eb2c82fb3aa0317e70dc710: Status 404 returned error can't find the container with id 3d2721c5f3ad01a8ac7ffc57a39df1872ace2b3a7eb2c82fb3aa0317e70dc710
Oct 30 19:37:31.188944 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:31.188926    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:37:31.527602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:31.527518    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 19:37:31.527602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:31.527585    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 19:37:31.820947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:31.820856    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc" event={"ID":"525f236d-76e8-4404-9c8c-5af6416906de","Type":"ContainerStarted","Data":"3d2721c5f3ad01a8ac7ffc57a39df1872ace2b3a7eb2c82fb3aa0317e70dc710"}
Oct 30 19:37:32.526840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:32.526740    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 19:37:32.526840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:32.526795    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 19:37:32.862218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:32.862166    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc" event={"ID":"525f236d-76e8-4404-9c8c-5af6416906de","Type":"ContainerStarted","Data":"939648a92fe113f2d37a189cbc12d58d78fec1cefc3f81d29ba81d6b5b7d6346"}
Oct 30 19:37:32.862550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:32.862507    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:32.982703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:32.982649    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc" podStartSLOduration=3.072994105 podStartE2EDuration="3.982633851s" totalImagesPullingTime="909.639746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:32.978918854 +0000 UTC m=+3045.563401852" watchObservedRunningTime="2026-10-30 19:37:32.982633851 +0000 UTC m=+3045.567116845"
Oct 30 19:37:33.527175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:33.527137    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 19:37:33.527659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:33.527197    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 19:37:33.880308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:33.880264    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"]
Oct 30 19:37:33.880558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:33.880528    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy" containerID="cri-o://9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d" gracePeriod=30
Oct 30 19:37:34.527767 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:34.527721    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 19:37:34.528214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:34.527800    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 19:37:35.531468 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:35.531435    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:35.531928 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:35.531711    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:35.539574 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:35.539546    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:37:38.898082 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:38.898059    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:37:39.311912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.311884    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.445895    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-envoy\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.445967    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-podinfo\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.446022    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-credential-socket\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.446057    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-socket\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.446100    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-data\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.446161    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-kube-api-access-9xmng\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-kube-api-access-9xmng\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.446207    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-token\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-token\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.446311    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istiod-ca-cert\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.447230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.446422    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-certs\") pod \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\" (UID: \"504ff7c3-69c5-4422-a4b2-ba43ecf1197f\") "
Oct 30 19:37:39.448235 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.447927    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-certs" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:39.449051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.449025    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-credential-socket" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:39.449452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.449432    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-data" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:39.449756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.449737    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-socket" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:39.451159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.451133    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istiod-ca-cert" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:39.485056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.484996    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-podinfo" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:37:39.485056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.485017    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-token" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:39.485401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.485356    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-envoy" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:39.487144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.487117    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-kube-api-access-9xmng" pod "504ff7c3-69c5-4422-a4b2-ba43ecf1197f" (UID: "504ff7c3-69c5-4422-a4b2-ba43ecf1197f"). InnerVolumeSpecName "kube-api-access-9xmng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:39.548468 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548428    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.548468 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548470    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.548707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548488    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.548707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548503    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.548707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548519    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.548707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548533    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.548707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548548    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.548707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548562    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xmng\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-kube-api-access-9xmng\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.548707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.548578    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504ff7c3-69c5-4422-a4b2-ba43ecf1197f-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.935994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.935960    5232 generic.go:356] "Generic (PLEG): container finished" podID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerID="9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d" exitCode=0
Oct 30 19:37:39.936434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.936009    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"
Oct 30 19:37:39.936434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.936055    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" event={"ID":"504ff7c3-69c5-4422-a4b2-ba43ecf1197f","Type":"ContainerDied","Data":"9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d"}
Oct 30 19:37:39.936434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.936118    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw" event={"ID":"504ff7c3-69c5-4422-a4b2-ba43ecf1197f","Type":"ContainerDied","Data":"2bbd11e3036b28c21196c4918e7391f50c6fbb03158181c760aac8c616372901"}
Oct 30 19:37:39.936434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.936146    5232 scope.go:122] "RemoveContainer" containerID="9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d"
Oct 30 19:37:39.957049 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.957010    5232 scope.go:122] "RemoveContainer" containerID="bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13"
Oct 30 19:37:39.982414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.982384    5232 scope.go:122] "RemoveContainer" containerID="9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d"
Oct 30 19:37:39.983208 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:39.982834    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d\": container with ID starting with 9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d not found: ID does not exist" containerID="9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d"
Oct 30 19:37:39.983208 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.982881    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d"} err="failed to get container status \"9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d\": rpc error: code = NotFound desc = could not find container \"9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d\": container with ID starting with 9031c1e5f4af0f2ffb5f03a4a7e7c8b9773e49085d64ee76b64c0bb56a22205d not found: ID does not exist"
Oct 30 19:37:39.983208 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.982910    5232 scope.go:122] "RemoveContainer" containerID="bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13"
Oct 30 19:37:39.983445 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:39.983227    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13\": container with ID starting with bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13 not found: ID does not exist" containerID="bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13"
Oct 30 19:37:39.983445 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:39.983267    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13"} err="failed to get container status \"bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13\": rpc error: code = NotFound desc = could not find container \"bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13\": container with ID starting with bb662204a6633b096856645bfdbd61e3a122f02bef58b6f8571857507d37fa13 not found: ID does not exist"
Oct 30 19:37:40.145909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:40.145870    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"]
Oct 30 19:37:40.150700 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:40.150675    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-83ac32-openshift-default-5d78bbccc-c5bhw"]
Oct 30 19:37:41.541202 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:41.541170    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" path="/var/lib/kubelet/pods/504ff7c3-69c5-4422-a4b2-ba43ecf1197f/volumes"
Oct 30 19:37:46.187246 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:46.187202    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:37:56.136384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.136351    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"]
Oct 30 19:37:56.137018 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.137000    5232 cpu_manager.go:414] "containerMap: removing container" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy"
Oct 30 19:37:56.137090 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.137020    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy"
Oct 30 19:37:56.137148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.137136    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy"
Oct 30 19:37:56.137184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.137156    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy"
Oct 30 19:37:56.141756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.141737    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.199032 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.194840    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-af72cf-openshift-default-dockercfg-pcwgc\""
Oct 30 19:37:56.272304 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.272273    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"]
Oct 30 19:37:56.279009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.278960    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81846c84-b541-44b0-84e3-f0063c18d613-istio-podinfo\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.279185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.279023    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-envoy\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.279185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.279063    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-istio-token\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.279185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.279166    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-socket\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.279318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.279196    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-data\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.279318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.279256    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvtj4\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-kube-api-access-cvtj4\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.279318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.279289    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81846c84-b541-44b0-84e3-f0063c18d613-istiod-ca-cert\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.279466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.279376    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-certs\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.279466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.279435    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-credential-socket\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.380888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.380772    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-socket\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.380888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.380845    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-data\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.380888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.380883    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvtj4\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-kube-api-access-cvtj4\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.380905    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81846c84-b541-44b0-84e3-f0063c18d613-istiod-ca-cert\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.380968    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-certs\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381013    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-credential-socket\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381075    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81846c84-b541-44b0-84e3-f0063c18d613-istio-podinfo\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381228    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-envoy\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381262    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-socket\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381293    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-data\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381300    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-istio-token\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381384    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-certs\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381463    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-credential-socket\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.381798 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.381778    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81846c84-b541-44b0-84e3-f0063c18d613-istiod-ca-cert\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.383468 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.383445    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-envoy\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.383699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.383681    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81846c84-b541-44b0-84e3-f0063c18d613-istio-podinfo\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.440280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.440202    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-istio-token\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.440413 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.440282    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvtj4\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-kube-api-access-cvtj4\") pod \"e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") " pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.455414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.455390    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:56.885118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.885067    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"]
Oct 30 19:37:56.889937 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:37:56.889908    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81846c84_b541_44b0_84e3_f0063c18d613.slice/crio-d901888c2c932d019ec7e46ef4d9b771b8015246ae1e67c205f215b360923843 WatchSource:0}: Error finding container d901888c2c932d019ec7e46ef4d9b771b8015246ae1e67c205f215b360923843: Status 404 returned error can't find the container with id d901888c2c932d019ec7e46ef4d9b771b8015246ae1e67c205f215b360923843
Oct 30 19:37:56.892662 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.892622    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:56.892750 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.892707    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:56.892750 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:56.892738    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:57.076518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:57.076472    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" event={"ID":"81846c84-b541-44b0-84e3-f0063c18d613","Type":"ContainerStarted","Data":"ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22"}
Oct 30 19:37:57.076713 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:57.076544    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" event={"ID":"81846c84-b541-44b0-84e3-f0063c18d613","Type":"ContainerStarted","Data":"d901888c2c932d019ec7e46ef4d9b771b8015246ae1e67c205f215b360923843"}
Oct 30 19:37:57.202749 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:57.202617    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" podStartSLOduration=1.202598267 podStartE2EDuration="1.202598267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:57.195414365 +0000 UTC m=+3069.779897364" watchObservedRunningTime="2026-10-30 19:37:57.202598267 +0000 UTC m=+3069.787081265"
Oct 30 19:37:57.455899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:57.455773    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:57.461374 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:57.461351    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:58.086863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.086835    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:58.087982 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.087962    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:37:58.735540 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.735506    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"]
Oct 30 19:37:58.736111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.736095    5232 cpu_manager.go:414] "containerMap: removing container" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy"
Oct 30 19:37:58.736163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.736113    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="504ff7c3-69c5-4422-a4b2-ba43ecf1197f" containerName="istio-proxy"
Oct 30 19:37:58.740052 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.740032    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.790166 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.790134    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-2186d8-serving-cert\""
Oct 30 19:37:58.799088 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.799056    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"]
Oct 30 19:37:58.814897 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.814861    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzt59\" (UniqueName: \"kubernetes.io/projected/a51573e1-e9cb-44ef-94d7-c25d887678a4-kube-api-access-wzt59\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.815163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.815132    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a51573e1-e9cb-44ef-94d7-c25d887678a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.815305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.815283    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.815420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.815400    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.916874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.916838    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzt59\" (UniqueName: \"kubernetes.io/projected/a51573e1-e9cb-44ef-94d7-c25d887678a4-kube-api-access-wzt59\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.917072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.916890    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a51573e1-e9cb-44ef-94d7-c25d887678a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.917072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.916954    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.917072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.917028    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.917243 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:58.917176    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-serving-cert: secret "maas-api-e2e-shared-a-w0-2186d8-serving-cert" not found
Oct 30 19:37:58.917285 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:37:58.917259    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls podName:a51573e1-e9cb-44ef-94d7-c25d887678a4 nodeName:}" failed. No retries permitted until 2026-10-30 19:37:59.417234708 +0000 UTC m=+3072.001717698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls") pod "maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr" (UID: "a51573e1-e9cb-44ef-94d7-c25d887678a4") : secret "maas-api-e2e-shared-a-w0-2186d8-serving-cert" not found
Oct 30 19:37:58.917650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.917628    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a51573e1-e9cb-44ef-94d7-c25d887678a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:58.919401 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:58.919380    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:59.008645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:59.008294    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzt59\" (UniqueName: \"kubernetes.io/projected/a51573e1-e9cb-44ef-94d7-c25d887678a4-kube-api-access-wzt59\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:59.425886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:59.425849    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:59.428318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:59.428292    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:37:59.653885 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:37:59.653846    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:38:00.029839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:00.029726    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"]
Oct 30 19:38:00.033498 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:38:00.033466    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51573e1_e9cb_44ef_94d7_c25d887678a4.slice/crio-480295a280ab295b733e1e726f730ea72f673e81c2f3dbf256522a5b8bc47822 WatchSource:0}: Error finding container 480295a280ab295b733e1e726f730ea72f673e81c2f3dbf256522a5b8bc47822: Status 404 returned error can't find the container with id 480295a280ab295b733e1e726f730ea72f673e81c2f3dbf256522a5b8bc47822
Oct 30 19:38:00.100263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:00.100226    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr" event={"ID":"a51573e1-e9cb-44ef-94d7-c25d887678a4","Type":"ContainerStarted","Data":"480295a280ab295b733e1e726f730ea72f673e81c2f3dbf256522a5b8bc47822"}
Oct 30 19:38:01.109389 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:01.109351    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr" event={"ID":"a51573e1-e9cb-44ef-94d7-c25d887678a4","Type":"ContainerStarted","Data":"d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029"}
Oct 30 19:38:01.109925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:01.109486    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:38:01.226799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:01.226672    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr" podStartSLOduration=2.343916263 podStartE2EDuration="3.226653787s" totalImagesPullingTime="882.737524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:01.223428381 +0000 UTC m=+3073.807911379" watchObservedRunningTime="2026-10-30 19:38:01.226653787 +0000 UTC m=+3073.811136785"
Oct 30 19:38:07.121833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:07.121782    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:38:08.939710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:08.939668    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"]
Oct 30 19:38:08.943932 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:08.943910    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:08.994116 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:08.994086    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-85f471-serving-cert\""
Oct 30 19:38:09.052257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.052222    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.052547 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.052352    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"]
Oct 30 19:38:09.052680 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.052526    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spsj9\" (UniqueName: \"kubernetes.io/projected/7d447874-7803-46a2-bcbe-e8a9bffb33d4-kube-api-access-spsj9\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.052746 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.052724    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d447874-7803-46a2-bcbe-e8a9bffb33d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.052880 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.052864    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.154130 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.154090    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.154340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.154147    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spsj9\" (UniqueName: \"kubernetes.io/projected/7d447874-7803-46a2-bcbe-e8a9bffb33d4-kube-api-access-spsj9\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.154340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.154177    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d447874-7803-46a2-bcbe-e8a9bffb33d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.154340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.154242    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.154551 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:38:09.154407    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-serving-cert: secret "maas-api-e2e-shared-a-w2-85f471-serving-cert" not found
Oct 30 19:38:09.154551 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:38:09.154479    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls podName:7d447874-7803-46a2-bcbe-e8a9bffb33d4 nodeName:}" failed. No retries permitted until 2026-10-30 19:38:09.654461204 +0000 UTC m=+3082.238944184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls") pod "maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4" (UID: "7d447874-7803-46a2-bcbe-e8a9bffb33d4") : secret "maas-api-e2e-shared-a-w2-85f471-serving-cert" not found
Oct 30 19:38:09.154942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.154921    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d447874-7803-46a2-bcbe-e8a9bffb33d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.156614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.156589    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.215578 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.215503    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spsj9\" (UniqueName: \"kubernetes.io/projected/7d447874-7803-46a2-bcbe-e8a9bffb33d4-kube-api-access-spsj9\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.661123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.661077    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.663623 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.663598    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:09.857513 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:09.857482    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:10.232080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:10.231974    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"]
Oct 30 19:38:10.236625 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:38:10.236598    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d447874_7803_46a2_bcbe_e8a9bffb33d4.slice/crio-b5e340df5f76c2e39b6bb0ae0f5df5ebc30324011d1d7db12658514718c74aa6 WatchSource:0}: Error finding container b5e340df5f76c2e39b6bb0ae0f5df5ebc30324011d1d7db12658514718c74aa6: Status 404 returned error can't find the container with id b5e340df5f76c2e39b6bb0ae0f5df5ebc30324011d1d7db12658514718c74aa6
Oct 30 19:38:11.178760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:11.178721    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4" event={"ID":"7d447874-7803-46a2-bcbe-e8a9bffb33d4","Type":"ContainerStarted","Data":"b5e340df5f76c2e39b6bb0ae0f5df5ebc30324011d1d7db12658514718c74aa6"}
Oct 30 19:38:12.194891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:12.194844    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4" event={"ID":"7d447874-7803-46a2-bcbe-e8a9bffb33d4","Type":"ContainerStarted","Data":"5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85"}
Oct 30 19:38:12.195325 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:12.195039    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:12.309609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:12.309543    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4" podStartSLOduration=3.34852821 podStartE2EDuration="4.309525733s" totalImagesPullingTime="960.997523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:12.308361608 +0000 UTC m=+3084.892844607" watchObservedRunningTime="2026-10-30 19:38:12.309525733 +0000 UTC m=+3084.894008732"
Oct 30 19:38:16.188516 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:38:16.188474    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:38:18.206986 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:18.206958    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:38:33.345489 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:33.345456    5232 generic.go:356] "Generic (PLEG): container finished" podID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerID="6224ebe6a7c20904a7e85aa0e355c9636de85ea3afc8a60498135accb5453b65" exitCode=0
Oct 30 19:38:33.345968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:33.345533    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" event={"ID":"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38","Type":"ContainerDied","Data":"6224ebe6a7c20904a7e85aa0e355c9636de85ea3afc8a60498135accb5453b65"}
Oct 30 19:38:33.346026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:33.345972    5232 scope.go:122] "RemoveContainer" containerID="6224ebe6a7c20904a7e85aa0e355c9636de85ea3afc8a60498135accb5453b65"
Oct 30 19:38:33.347421 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:33.347387    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:33.347548 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:33.347472    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:33.347548 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:33.347523    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:34.366908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:34.366877    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" event={"ID":"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38","Type":"ContainerStarted","Data":"f0c51cf12ed86f68af9443a861e7d5ee8d9d5e4a151bcd4d6a9a4e087950710d"}
Oct 30 19:38:35.035706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:35.035669    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:38:35.037212 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:35.037177    5232 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:38:35.037543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:35.037501    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 19:38:36.035669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:36.035631    5232 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:38:36.036222 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:36.035701    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 19:38:36.752846 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:36.747935    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:38:37.035888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:37.035778    5232 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:38:37.035888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:37.035856    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 19:38:38.036027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:38.035994    5232 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:38:38.036473 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:38.036057    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 19:38:39.035391 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:39.035352    5232 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:38:39.035590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:39.035414    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 30 19:38:40.040430 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:40.040397    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:38:40.041346 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:40.041329    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:38:46.189179 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:38:46.189136    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:38:47.895425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:47.895388    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"]
Oct 30 19:38:47.898631 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:47.898614    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:47.946068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:47.946033    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:47.946249 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:47.946095    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jcsn\" (UniqueName: \"kubernetes.io/projected/51302f02-327c-45bc-9d51-5e11a5892b22-kube-api-access-9jcsn\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:47.946249 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:47.946176    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-metrics-certs\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:47.946369 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:47.946296    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51302f02-327c-45bc-9d51-5e11a5892b22-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:47.949266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:47.949243    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-af72cf-serving-cert\""
Oct 30 19:38:48.001712 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.001682    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"]
Oct 30 19:38:48.061842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.058305    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51302f02-327c-45bc-9d51-5e11a5892b22-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.061842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.058547    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.061842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.058628    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jcsn\" (UniqueName: \"kubernetes.io/projected/51302f02-327c-45bc-9d51-5e11a5892b22-kube-api-access-9jcsn\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.061842 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.058655    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-metrics-certs\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.075045 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:38:48.075007    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-serving-cert: secret "maas-api-e2e-praxis-a-af72cf-serving-cert" not found
Oct 30 19:38:48.075240 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:38:48.075102    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls podName:51302f02-327c-45bc-9d51-5e11a5892b22 nodeName:}" failed. No retries permitted until 2026-10-30 19:38:48.575078705 +0000 UTC m=+3121.159561703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls") pod "maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b" (UID: "51302f02-327c-45bc-9d51-5e11a5892b22") : secret "maas-api-e2e-praxis-a-af72cf-serving-cert" not found
Oct 30 19:38:48.076586 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.076556    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51302f02-327c-45bc-9d51-5e11a5892b22-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.079512 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.079487    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-metrics-certs\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.135648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.135614    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jcsn\" (UniqueName: \"kubernetes.io/projected/51302f02-327c-45bc-9d51-5e11a5892b22-kube-api-access-9jcsn\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.665144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.665103    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.667473 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.667447    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls\") pod \"maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:48.812003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:48.811971    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:49.196732 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:49.196698    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"]
Oct 30 19:38:49.200402 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:38:49.200375    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51302f02_327c_45bc_9d51_5e11a5892b22.slice/crio-791b4fa355b44fa7e99211b8afd0651f030c820d10f9d37e34ed87f09aef1cc5 WatchSource:0}: Error finding container 791b4fa355b44fa7e99211b8afd0651f030c820d10f9d37e34ed87f09aef1cc5: Status 404 returned error can't find the container with id 791b4fa355b44fa7e99211b8afd0651f030c820d10f9d37e34ed87f09aef1cc5
Oct 30 19:38:49.492310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:49.492229    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b" event={"ID":"51302f02-327c-45bc-9d51-5e11a5892b22","Type":"ContainerStarted","Data":"791b4fa355b44fa7e99211b8afd0651f030c820d10f9d37e34ed87f09aef1cc5"}
Oct 30 19:38:50.501070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:50.500984    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b" event={"ID":"51302f02-327c-45bc-9d51-5e11a5892b22","Type":"ContainerStarted","Data":"5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1"}
Oct 30 19:38:50.501432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:50.501159    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:38:50.621590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:50.621521    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b" podStartSLOduration=2.684335636 podStartE2EDuration="3.62150178s" totalImagesPullingTime="937.166144ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:50.616143474 +0000 UTC m=+3123.200626468" watchObservedRunningTime="2026-10-30 19:38:50.62150178 +0000 UTC m=+3123.205984779"
Oct 30 19:38:56.513539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:38:56.513507    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:39:11.756845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.753773    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"]
Oct 30 19:39:11.765911 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.765122    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.822154 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.822117    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-c01685-openshift-default-dockercfg-9k28m\""
Oct 30 19:39:11.839713 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.839672    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed138b6-03f5-460e-bd64-067bba10054f-istio-podinfo\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.840029 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.840001    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-certs\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.840248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.840230    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-credential-socket\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.840368 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.840347    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-data\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.840551 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.840533    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-istio-token\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.840799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.840781    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed138b6-03f5-460e-bd64-067bba10054f-istiod-ca-cert\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.841026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.841007    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-socket\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.841165 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.841151    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-envoy\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.841276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.841261    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7wpk\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-kube-api-access-d7wpk\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.908443 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.908359    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"]
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962258    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-credential-socket\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962306    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-data\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962392    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-istio-token\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962478    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed138b6-03f5-460e-bd64-067bba10054f-istiod-ca-cert\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962572    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-socket\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962624    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-envoy\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962656    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7wpk\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-kube-api-access-d7wpk\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962693    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed138b6-03f5-460e-bd64-067bba10054f-istio-podinfo\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.962728    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-certs\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.963288    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-certs\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.963568    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-credential-socket\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.963829    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-data\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.964840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.964790    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed138b6-03f5-460e-bd64-067bba10054f-istiod-ca-cert\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.989939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.987676    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-socket\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.991362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.991325    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed138b6-03f5-460e-bd64-067bba10054f-istio-podinfo\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:11.994642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:11.994616    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-envoy\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:12.025385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:12.025281    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-istio-token\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:12.067546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:12.067416    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7wpk\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-kube-api-access-d7wpk\") pod \"e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") " pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:12.143837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:12.142063    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:12.650736 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:39:12.650705    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed138b6_03f5_460e_bd64_067bba10054f.slice/crio-67ade79c5f4962f8b138aa7950b28da145a81bf68d129dc61deacf54be681562 WatchSource:0}: Error finding container 67ade79c5f4962f8b138aa7950b28da145a81bf68d129dc61deacf54be681562: Status 404 returned error can't find the container with id 67ade79c5f4962f8b138aa7950b28da145a81bf68d129dc61deacf54be681562
Oct 30 19:39:12.653600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:12.653552    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:12.653730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:12.653644    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:12.653730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:12.653686    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:12.656837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:12.656800    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"]
Oct 30 19:39:12.665567 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:12.665539    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" event={"ID":"0ed138b6-03f5-460e-bd64-067bba10054f","Type":"ContainerStarted","Data":"67ade79c5f4962f8b138aa7950b28da145a81bf68d129dc61deacf54be681562"}
Oct 30 19:39:13.676517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:13.676473    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" event={"ID":"0ed138b6-03f5-460e-bd64-067bba10054f","Type":"ContainerStarted","Data":"8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b"}
Oct 30 19:39:13.844825 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:13.844750    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" podStartSLOduration=2.844735524 podStartE2EDuration="2.844735524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:13.843940866 +0000 UTC m=+3146.428423863" watchObservedRunningTime="2026-10-30 19:39:13.844735524 +0000 UTC m=+3146.429218520"
Oct 30 19:39:14.129025 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:14.128984    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:39:14.129226 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:14.129072    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:39:14.142805 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:14.142771    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:14.149163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:14.149131    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:14.683282 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:14.683247    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:14.684473 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:14.684449    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:39:16.187229 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:39:16.187194    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:39:16.697919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:16.697889    5232 generic.go:356] "Generic (PLEG): container finished" podID="81846c84-b541-44b0-84e3-f0063c18d613" containerID="ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22" exitCode=0
Oct 30 19:39:16.698148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:16.697971    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" event={"ID":"81846c84-b541-44b0-84e3-f0063c18d613","Type":"ContainerDied","Data":"ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22"}
Oct 30 19:39:16.698542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:16.698523    5232 scope.go:122] "RemoveContainer" containerID="ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22"
Oct 30 19:39:16.700007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:16.699981    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:16.700071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:16.700044    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:16.700112 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:16.700076    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:17.707087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:17.707051    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" event={"ID":"81846c84-b541-44b0-84e3-f0063c18d613","Type":"ContainerStarted","Data":"0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709"}
Oct 30 19:39:18.462031 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:18.461995    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:39:18.462258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:18.462219    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:39:18.462341 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:18.462276    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Oct 30 19:39:19.461995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:19.461950    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:39:19.462501 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:19.462015    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Oct 30 19:39:20.466838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:20.460253    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:39:20.466838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:20.460318    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Oct 30 19:39:21.467534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:21.463138    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:39:21.467534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:21.463215    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Oct 30 19:39:22.456199 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:22.456160    5232 patch_prober.go:29] interesting pod/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:39:22.456509 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:22.456482    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Oct 30 19:39:23.459951 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:23.459917    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:39:23.460482 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:23.460465    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:39:23.461162 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:23.461141    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:39:37.277610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.277535    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"]
Oct 30 19:39:37.282760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.282741    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.333040 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.333010    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-59594f-openshift-default-dockercfg-hb2q6\""
Oct 30 19:39:37.384043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.384014    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"]
Oct 30 19:39:37.410628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.410593    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-data\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.410791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.410703    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-socket\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.410791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.410742    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istiod-ca-cert\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.410791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.410761    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-envoy\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.410937 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.410904    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qtbn\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-kube-api-access-8qtbn\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.410978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.410941    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-podinfo\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.410978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.410963    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-credential-socket\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.411094 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.411082    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-token\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.411134 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.411124    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-certs\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.518659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.518143    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-socket\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.519595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.518922    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istiod-ca-cert\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.519762 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.519745    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-envoy\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.520073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.520055    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istiod-ca-cert\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.520269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.520192    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qtbn\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-kube-api-access-8qtbn\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.520426 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.520405    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-podinfo\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.520696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.520679    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-credential-socket\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.520858 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.518617    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-socket\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.521308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.521183    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-token\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.521502 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.521245    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-credential-socket\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.521618 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.521600    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-certs\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.521774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.521756    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-data\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.526866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.522152    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-data\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.526866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.522299    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-certs\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.526866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.525088    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-envoy\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.530395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.530315    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-podinfo\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.585087 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.585051    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-token\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.585492 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.585473    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qtbn\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-kube-api-access-8qtbn\") pod \"e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") " pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:37.599052 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:37.599024    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:38.042973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:38.042932    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"]
Oct 30 19:39:38.047447 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:39:38.047417    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0b309f_e728_4e7d_8ca8_1b3bd8e8a7ab.slice/crio-0cff19b068614768a44d7aa7f42f58700089d9857a8187db984ee9f4e188ffeb WatchSource:0}: Error finding container 0cff19b068614768a44d7aa7f42f58700089d9857a8187db984ee9f4e188ffeb: Status 404 returned error can't find the container with id 0cff19b068614768a44d7aa7f42f58700089d9857a8187db984ee9f4e188ffeb
Oct 30 19:39:38.050070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:38.050035    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:38.050137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:38.050109    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:38.050174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:38.050140    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:38.860904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:38.860870    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" event={"ID":"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab","Type":"ContainerStarted","Data":"9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958"}
Oct 30 19:39:38.861327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:38.860932    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" event={"ID":"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab","Type":"ContainerStarted","Data":"0cff19b068614768a44d7aa7f42f58700089d9857a8187db984ee9f4e188ffeb"}
Oct 30 19:39:38.980488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:38.980432    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" podStartSLOduration=1.980417164 podStartE2EDuration="1.980417164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:38.977699195 +0000 UTC m=+3171.562182193" watchObservedRunningTime="2026-10-30 19:39:38.980417164 +0000 UTC m=+3171.564900232"
Oct 30 19:39:39.600119 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:39.600086    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:39.606276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:39.606250    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:39.867276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:39.867196    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:39.868037 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:39:39.868013    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:39:46.187703 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:39:46.187666    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:40:05.811462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:05.811419    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"]
Oct 30 19:40:05.816100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:05.816075    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:05.867005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:05.866974    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-d6f58f-serving-cert\""
Oct 30 19:40:05.887403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:05.887362    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"]
Oct 30 19:40:05.939345 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:05.939272    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:05.939345 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:05.939329    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vj4x\" (UniqueName: \"kubernetes.io/projected/cae98346-56e2-4e97-ad8f-97b617d91367-kube-api-access-5vj4x\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:05.939645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:05.939530    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cae98346-56e2-4e97-ad8f-97b617d91367-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:05.939714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:05.939650    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.041053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.041003    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cae98346-56e2-4e97-ad8f-97b617d91367-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.041262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.041096    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.041262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.041207    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.041262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.041237    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vj4x\" (UniqueName: \"kubernetes.io/projected/cae98346-56e2-4e97-ad8f-97b617d91367-kube-api-access-5vj4x\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.041262 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:40:06.041244    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-serving-cert: secret "maas-api-e2e-shared-b-w0-d6f58f-serving-cert" not found
Oct 30 19:40:06.041503 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:40:06.041319    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls podName:cae98346-56e2-4e97-ad8f-97b617d91367 nodeName:}" failed. No retries permitted until 2026-10-30 19:40:06.541300662 +0000 UTC m=+3199.125783663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls") pod "maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26" (UID: "cae98346-56e2-4e97-ad8f-97b617d91367") : secret "maas-api-e2e-shared-b-w0-d6f58f-serving-cert" not found
Oct 30 19:40:06.041791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.041770    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cae98346-56e2-4e97-ad8f-97b617d91367-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.043707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.043681    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.105517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.103978    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vj4x\" (UniqueName: \"kubernetes.io/projected/cae98346-56e2-4e97-ad8f-97b617d91367-kube-api-access-5vj4x\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.549643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.549543    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.552075 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.552052    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:06.730588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:06.730544    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:07.119719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:07.119681    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"]
Oct 30 19:40:07.123421 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:40:07.123391    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae98346_56e2_4e97_ad8f_97b617d91367.slice/crio-4cde925fbfd8152354e40984ded49f74e35ab0eaafc75c4856da8b7fe8c80819 WatchSource:0}: Error finding container 4cde925fbfd8152354e40984ded49f74e35ab0eaafc75c4856da8b7fe8c80819: Status 404 returned error can't find the container with id 4cde925fbfd8152354e40984ded49f74e35ab0eaafc75c4856da8b7fe8c80819
Oct 30 19:40:08.072012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:08.071976    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26" event={"ID":"cae98346-56e2-4e97-ad8f-97b617d91367","Type":"ContainerStarted","Data":"4cde925fbfd8152354e40984ded49f74e35ab0eaafc75c4856da8b7fe8c80819"}
Oct 30 19:40:09.080397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:09.080356    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26" event={"ID":"cae98346-56e2-4e97-ad8f-97b617d91367","Type":"ContainerStarted","Data":"c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a"}
Oct 30 19:40:09.080870 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:09.080425    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:09.195304 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:09.195246    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26" podStartSLOduration=3.16960038 podStartE2EDuration="4.195232692s" totalImagesPullingTime="1.025632312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:09.194397091 +0000 UTC m=+3201.778880088" watchObservedRunningTime="2026-10-30 19:40:09.195232692 +0000 UTC m=+3201.779715688"
Oct 30 19:40:15.095634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:15.095600    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:40:16.193027 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:40:16.192983    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:40:46.188275 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:40:46.188229    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:40:48.434338 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.434307    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"]
Oct 30 19:40:48.458136 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.458102    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.498297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.498260    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"]
Oct 30 19:40:48.561833 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.561036    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-59594f-serving-cert\""
Oct 30 19:40:48.570706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.570675    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv2gq\" (UniqueName: \"kubernetes.io/projected/9dbdae8e-cabf-4056-8f45-550d43dafa5f-kube-api-access-kv2gq\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.570925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.570898    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.570994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.570954    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.571141 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.571110    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbdae8e-cabf-4056-8f45-550d43dafa5f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.672598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.672554    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.672924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.672885    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.672924 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:40:48.672728    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-serving-cert: secret "maas-api-e2e-shared-b-w1-59594f-serving-cert" not found
Oct 30 19:40:48.673120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.672973    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbdae8e-cabf-4056-8f45-550d43dafa5f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.673120 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:40:48.673003    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls podName:9dbdae8e-cabf-4056-8f45-550d43dafa5f nodeName:}" failed. No retries permitted until 2026-10-30 19:40:49.172977028 +0000 UTC m=+3241.757460017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls") pod "maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd" (UID: "9dbdae8e-cabf-4056-8f45-550d43dafa5f") : secret "maas-api-e2e-shared-b-w1-59594f-serving-cert" not found
Oct 30 19:40:48.673261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.673182    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv2gq\" (UniqueName: \"kubernetes.io/projected/9dbdae8e-cabf-4056-8f45-550d43dafa5f-kube-api-access-kv2gq\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.673727 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.673705    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbdae8e-cabf-4056-8f45-550d43dafa5f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.675957 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.675935    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:48.775848 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:48.773537    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv2gq\" (UniqueName: \"kubernetes.io/projected/9dbdae8e-cabf-4056-8f45-550d43dafa5f-kube-api-access-kv2gq\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:49.188500 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:49.188464    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:49.191626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:49.191595    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:49.375559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:49.375526    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:49.769650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:49.769567    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"]
Oct 30 19:40:49.773994 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:40:49.773961    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dbdae8e_cabf_4056_8f45_550d43dafa5f.slice/crio-b90281845db676051ee79e2636648e764ca174d5ed2ef99f61c291566179595f WatchSource:0}: Error finding container b90281845db676051ee79e2636648e764ca174d5ed2ef99f61c291566179595f: Status 404 returned error can't find the container with id b90281845db676051ee79e2636648e764ca174d5ed2ef99f61c291566179595f
Oct 30 19:40:50.393168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:50.393135    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd" event={"ID":"9dbdae8e-cabf-4056-8f45-550d43dafa5f","Type":"ContainerStarted","Data":"b90281845db676051ee79e2636648e764ca174d5ed2ef99f61c291566179595f"}
Oct 30 19:40:51.404195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:51.404155    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd" event={"ID":"9dbdae8e-cabf-4056-8f45-550d43dafa5f","Type":"ContainerStarted","Data":"8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2"}
Oct 30 19:40:51.404685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:51.404264    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:57.417051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:57.417020    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:40:57.447296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:57.447267    5232 generic.go:356] "Generic (PLEG): container finished" podID="0ed138b6-03f5-460e-bd64-067bba10054f" containerID="8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b" exitCode=0
Oct 30 19:40:57.447469 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:57.447344    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" event={"ID":"0ed138b6-03f5-460e-bd64-067bba10054f","Type":"ContainerDied","Data":"8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b"}
Oct 30 19:40:57.447731 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:57.447716    5232 scope.go:122] "RemoveContainer" containerID="8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b"
Oct 30 19:40:57.449285 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:57.449257    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:57.449356 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:57.449324    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:57.449394 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:57.449359    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:57.535964 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:57.535914    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd" podStartSLOduration=8.692240985 podStartE2EDuration="9.535898946s" totalImagesPullingTime="843.657961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:51.564679905 +0000 UTC m=+3244.149162903" watchObservedRunningTime="2026-10-30 19:40:57.535898946 +0000 UTC m=+3250.120381933"
Oct 30 19:40:58.469989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:58.469947    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" event={"ID":"0ed138b6-03f5-460e-bd64-067bba10054f","Type":"ContainerStarted","Data":"a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82"}
Oct 30 19:40:59.156912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:59.156873    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:40:59.157124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:59.156942    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:40:59.157124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:59.156999    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 19:40:59.684122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:40:59.684094    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:41:00.157837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:00.150239    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:41:00.157837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:00.150307    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 19:41:01.155261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:01.155219    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:41:01.156836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:01.156780    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 19:41:02.149289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:02.149248    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:41:02.149596 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:02.149568    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 19:41:03.152285 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:03.152239    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:41:03.162035 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:03.161967    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 19:41:04.147414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:04.147382    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:41:04.148312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:04.148290    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 19:41:16.190736 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:41:16.190683    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:41:24.688247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:24.688150    5232 generic.go:356] "Generic (PLEG): container finished" podID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerID="9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958" exitCode=0
Oct 30 19:41:24.688247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:24.688218    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" event={"ID":"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab","Type":"ContainerDied","Data":"9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958"}
Oct 30 19:41:24.688719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:24.688688    5232 scope.go:122] "RemoveContainer" containerID="9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958"
Oct 30 19:41:24.690341 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:24.690311    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:24.690450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:24.690391    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:24.690450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:24.690441    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:24.868153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:24.868115    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:41:25.698768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:25.698727    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" event={"ID":"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab","Type":"ContainerStarted","Data":"683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46"}
Oct 30 19:41:26.600147 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:26.600110    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:41:26.601501 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:26.601475    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:41:26.601626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:26.601524    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:41:27.599611 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:27.599574    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:41:27.600056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:27.599629    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:41:28.599610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:28.599572    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:41:28.600143 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:28.599635    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:41:29.600126 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:29.600090    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:41:29.600545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:29.600142    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:41:30.600372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:30.600336    5232 patch_prober.go:29] interesting pod/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:41:30.600791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:30.600396    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:41:31.605001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:31.604969    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:41:31.605445 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:31.605425    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:41:31.606237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:31.606214    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:41:46.206010 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:41:46.205967    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:41:48.058254 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:48.058226    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:41:48.062699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:48.062676    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:41:48.075218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:48.075196    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:41:48.079383 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:41:48.079365    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:42:09.234140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:09.234102    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"]
Oct 30 19:42:09.234664 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:09.234331    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy" containerID="cri-o://62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5" gracePeriod=30
Oct 30 19:42:09.281551 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:09.281514    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"]
Oct 30 19:42:09.281771 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:09.281732    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy" containerID="cri-o://2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4" gracePeriod=30
Oct 30 19:42:09.285029 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:09.284999    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"]
Oct 30 19:42:09.285262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:09.285217    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy" containerID="cri-o://9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5" gracePeriod=30
Oct 30 19:42:09.328885 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:09.328833    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"]
Oct 30 19:42:09.329082 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:09.329059    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy" containerID="cri-o://ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92" gracePeriod=30
Oct 30 19:42:14.588877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.588849    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:42:14.630450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.630405    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istiod-ca-cert\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.630640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.630542    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-podinfo\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.630640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.630583    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-kube-api-access-phvtz\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-kube-api-access-phvtz\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.630638    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-socket\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.630744    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-certs\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.630774    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-credential-socket\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.630808    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-envoy\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.630887    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-token\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-token\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.631530    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-certs" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.631577    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-credential-socket" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.631896    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istiod-ca-cert" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:14.632061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.631961    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-socket" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.636088 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.635747    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-token" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:14.637335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.637248    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-data\") pod \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\" (UID: \"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d\") "
Oct 30 19:42:14.639243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.637583    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-data" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.639243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.638007    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.639243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.638033    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.639243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.638048    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.639243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.638063    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.639243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.638078    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.639243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.638091    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.646633 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.646592    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-envoy" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.663866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.655000    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-podinfo" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:42:14.663866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.658311    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-kube-api-access-phvtz" pod "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" (UID: "b18f59f9-7bd1-4691-b69e-9ab4e4134f5d"). InnerVolumeSpecName "kube-api-access-phvtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:14.701265 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.701125    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:42:14.711435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.711407    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:42:14.714993 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.714972    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:42:14.738533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738497    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-socket\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.738669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738573    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-envoy\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.738669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738605    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/691fea7b-46d8-4d38-878b-14623681a293-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/691fea7b-46d8-4d38-878b-14623681a293-istio-podinfo\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.738669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738633    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istiod-ca-cert\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.738669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738659    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-data\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.738904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738716    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-token\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-token\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.738904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738769    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-certs\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.738904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738777    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-socket" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.738904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738803    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-istio-token\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-istio-token\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.738904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738853    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-kube-api-access-4pqcg\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-kube-api-access-4pqcg\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.739247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738910    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-certs\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.739247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.738975    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-data\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.739247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739017    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e20ab810-41dd-465a-8883-08f583b528ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20ab810-41dd-465a-8883-08f583b528ee-istio-podinfo\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.739247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739047    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-credential-socket\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.739247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739090    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-kube-api-access-6fh4q\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-kube-api-access-6fh4q\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.739247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739150    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-credential-socket\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.739247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739129    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-data" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.739247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739178    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-socket\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.739721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739253    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-certs" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.739721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739268    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e20ab810-41dd-465a-8883-08f583b528ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20ab810-41dd-465a-8883-08f583b528ee-istiod-ca-cert\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.739721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739343    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-podinfo\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.739721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739384    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-data\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.739721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739611    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istiod-ca-cert" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:14.739721 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739617    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20ab810-41dd-465a-8883-08f583b528ee-istiod-ca-cert" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:14.740077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739431    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-kube-api-access-xglzz\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-kube-api-access-xglzz\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.740077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739765    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/691fea7b-46d8-4d38-878b-14623681a293-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/691fea7b-46d8-4d38-878b-14623681a293-istiod-ca-cert\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.740077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739805    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-envoy\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.740077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739872    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-envoy\") pod \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\" (UID: \"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7\") "
Oct 30 19:42:14.740077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739900    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-credential-socket\") pod \"691fea7b-46d8-4d38-878b-14623681a293\" (UID: \"691fea7b-46d8-4d38-878b-14623681a293\") "
Oct 30 19:42:14.740077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.739953    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-socket\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.740077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740021    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-certs\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.740077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740054    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-istio-token\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-istio-token\") pod \"e20ab810-41dd-465a-8883-08f583b528ee\" (UID: \"e20ab810-41dd-465a-8883-08f583b528ee\") "
Oct 30 19:42:14.740751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740728    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20ab810-41dd-465a-8883-08f583b528ee-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.740863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740759    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.740863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740777    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phvtz\" (UniqueName: \"kubernetes.io/projected/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-kube-api-access-phvtz\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.740863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740795    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.740863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740827    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.740863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740844    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.740863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740860    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.741221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.740874    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.741221 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.741205    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-certs" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.741358 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.741333    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-data" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.741910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.741356    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-token" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:14.741910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.741732    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-socket" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.741910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.741894    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-credential-socket" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.742117 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.742000    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-credential-socket" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.742117 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.742078    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-socket" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.742266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.742249    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-credential-socket" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.742333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.742263    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-data" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.742417 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.742394    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-certs" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.742757 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.742729    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/691fea7b-46d8-4d38-878b-14623681a293-istiod-ca-cert" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:14.743335 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.743279    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-istio-token" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:14.743451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.743369    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-envoy" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.743564 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.743528    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-istio-token" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:14.743992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.743959    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-kube-api-access-4pqcg" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "kube-api-access-4pqcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:14.744216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.744187    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/691fea7b-46d8-4d38-878b-14623681a293-istio-podinfo" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:42:14.744305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.744259    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-envoy" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.744516 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.744498    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-kube-api-access-6fh4q" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "kube-api-access-6fh4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:14.744626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.744610    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e20ab810-41dd-465a-8883-08f583b528ee-istio-podinfo" pod "e20ab810-41dd-465a-8883-08f583b528ee" (UID: "e20ab810-41dd-465a-8883-08f583b528ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:42:14.744804 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.744783    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-envoy" pod "691fea7b-46d8-4d38-878b-14623681a293" (UID: "691fea7b-46d8-4d38-878b-14623681a293"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:14.744804 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.744784    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-kube-api-access-xglzz" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "kube-api-access-xglzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:14.745995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.745977    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-podinfo" pod "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" (UID: "e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:42:14.842333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842301    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842333 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842336    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20ab810-41dd-465a-8883-08f583b528ee-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842347    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842359    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fh4q\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-kube-api-access-6fh4q\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842369    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842379    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842387    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842396    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842404    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xglzz\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-kube-api-access-xglzz\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842415    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/691fea7b-46d8-4d38-878b-14623681a293-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842425    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842434    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842443    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842453    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842462    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842471    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842478    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20ab810-41dd-465a-8883-08f583b528ee-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842486    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/691fea7b-46d8-4d38-878b-14623681a293-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842494    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842502    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/691fea7b-46d8-4d38-878b-14623681a293-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842510    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/691fea7b-46d8-4d38-878b-14623681a293-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:14.842539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:14.842518    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pqcg\" (UniqueName: \"kubernetes.io/projected/e20ab810-41dd-465a-8883-08f583b528ee-kube-api-access-4pqcg\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:15.065979 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.065944    5232 generic.go:356] "Generic (PLEG): container finished" podID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerID="62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5" exitCode=0
Oct 30 19:42:15.066175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.066011    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"
Oct 30 19:42:15.066175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.066023    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" event={"ID":"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d","Type":"ContainerDied","Data":"62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5"}
Oct 30 19:42:15.066175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.066099    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj" event={"ID":"b18f59f9-7bd1-4691-b69e-9ab4e4134f5d","Type":"ContainerDied","Data":"5b0fadeaede4aed2d1a6d550201809140db0ee6faef2f27c6cab4ac98233d766"}
Oct 30 19:42:15.066175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.066133    5232 scope.go:122] "RemoveContainer" containerID="62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5"
Oct 30 19:42:15.067881 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.067856    5232 generic.go:356] "Generic (PLEG): container finished" podID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerID="ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92" exitCode=0
Oct 30 19:42:15.068026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.067917    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"
Oct 30 19:42:15.068026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.067935    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" event={"ID":"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7","Type":"ContainerDied","Data":"ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92"}
Oct 30 19:42:15.068026 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.067987    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7" event={"ID":"e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7","Type":"ContainerDied","Data":"3e313d6513a928027b8b408b24ac3939f4c9407eece5b58a2908f1108cf78dba"}
Oct 30 19:42:15.069908 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.069885    5232 generic.go:356] "Generic (PLEG): container finished" podID="e20ab810-41dd-465a-8883-08f583b528ee" containerID="9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5" exitCode=0
Oct 30 19:42:15.070013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.069933    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"
Oct 30 19:42:15.070013 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.069974    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" event={"ID":"e20ab810-41dd-465a-8883-08f583b528ee","Type":"ContainerDied","Data":"9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5"}
Oct 30 19:42:15.070111 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.070023    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b" event={"ID":"e20ab810-41dd-465a-8883-08f583b528ee","Type":"ContainerDied","Data":"816e90bc9218dd8c8ed76c8badd53be33d0f50297587a3bbf86cf2a64d81a204"}
Oct 30 19:42:15.074479 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.074453    5232 generic.go:356] "Generic (PLEG): container finished" podID="691fea7b-46d8-4d38-878b-14623681a293" containerID="2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4" exitCode=0
Oct 30 19:42:15.074588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.074492    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" event={"ID":"691fea7b-46d8-4d38-878b-14623681a293","Type":"ContainerDied","Data":"2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4"}
Oct 30 19:42:15.074588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.074530    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q" event={"ID":"691fea7b-46d8-4d38-878b-14623681a293","Type":"ContainerDied","Data":"335996374f7ae5fbe6053d64940c552c9d481916a9a6e509e44587c4c0cdaf0a"}
Oct 30 19:42:15.074588 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.074493    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"
Oct 30 19:42:15.080321 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.080302    5232 scope.go:122] "RemoveContainer" containerID="62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5"
Oct 30 19:42:15.080633 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:42:15.080611    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5\": container with ID starting with 62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5 not found: ID does not exist" containerID="62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5"
Oct 30 19:42:15.080707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.080645    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5"} err="failed to get container status \"62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5\": rpc error: code = NotFound desc = could not find container \"62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5\": container with ID starting with 62042282a2db42d8944bc33a2f8bc9ad9fe5255e6825d226471cee637c4b5cf5 not found: ID does not exist"
Oct 30 19:42:15.080707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.080668    5232 scope.go:122] "RemoveContainer" containerID="ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92"
Oct 30 19:42:15.090848 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.090832    5232 scope.go:122] "RemoveContainer" containerID="ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92"
Oct 30 19:42:15.091140 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:42:15.091118    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92\": container with ID starting with ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92 not found: ID does not exist" containerID="ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92"
Oct 30 19:42:15.091233 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.091144    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92"} err="failed to get container status \"ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92\": rpc error: code = NotFound desc = could not find container \"ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92\": container with ID starting with ca7377e4857e495bb8e22110b374262fe362e55448272115d036d9e0e4268c92 not found: ID does not exist"
Oct 30 19:42:15.091233 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.091160    5232 scope.go:122] "RemoveContainer" containerID="9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5"
Oct 30 19:42:15.100595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.100576    5232 scope.go:122] "RemoveContainer" containerID="5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d"
Oct 30 19:42:15.113878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.113847    5232 scope.go:122] "RemoveContainer" containerID="9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5"
Oct 30 19:42:15.114243 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:42:15.114216    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5\": container with ID starting with 9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5 not found: ID does not exist" containerID="9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5"
Oct 30 19:42:15.114312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.114256    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5"} err="failed to get container status \"9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5\": rpc error: code = NotFound desc = could not find container \"9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5\": container with ID starting with 9d8ca2300492676e3c4dbbfdb85874508fa637d97dfa61b862cd86c9daaa62a5 not found: ID does not exist"
Oct 30 19:42:15.114312 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.114277    5232 scope.go:122] "RemoveContainer" containerID="5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d"
Oct 30 19:42:15.114554 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:42:15.114536    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d\": container with ID starting with 5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d not found: ID does not exist" containerID="5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d"
Oct 30 19:42:15.114600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.114562    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d"} err="failed to get container status \"5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d\": rpc error: code = NotFound desc = could not find container \"5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d\": container with ID starting with 5ed96c05bcba3b6907c4fbc7a0d34879a11fa0953db69b37232a770dcdd1817d not found: ID does not exist"
Oct 30 19:42:15.114600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.114579    5232 scope.go:122] "RemoveContainer" containerID="2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4"
Oct 30 19:42:15.124996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.124972    5232 scope.go:122] "RemoveContainer" containerID="2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4"
Oct 30 19:42:15.125313 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:42:15.125286    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4\": container with ID starting with 2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4 not found: ID does not exist" containerID="2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4"
Oct 30 19:42:15.125411 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.125319    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4"} err="failed to get container status \"2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4\": rpc error: code = NotFound desc = could not find container \"2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4\": container with ID starting with 2fc32585bc96722c5fcd4c7df07cdea84ac5ca52c9fd0e43fe9c48ff6c1741d4 not found: ID does not exist"
Oct 30 19:42:15.228799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.228763    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"]
Oct 30 19:42:15.251296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.251245    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-bg9vj"]
Oct 30 19:42:15.408422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.408341    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"]
Oct 30 19:42:15.414006 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.413977    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-j8k5q"]
Oct 30 19:42:15.521684 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.521645    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"]
Oct 30 19:42:15.524995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.524966    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-65kh7"]
Oct 30 19:42:15.541244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.541211    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="691fea7b-46d8-4d38-878b-14623681a293" path="/var/lib/kubelet/pods/691fea7b-46d8-4d38-878b-14623681a293/volumes"
Oct 30 19:42:15.541789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.541768    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" path="/var/lib/kubelet/pods/b18f59f9-7bd1-4691-b69e-9ab4e4134f5d/volumes"
Oct 30 19:42:15.542308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.542289    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" path="/var/lib/kubelet/pods/e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7/volumes"
Oct 30 19:42:15.628258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.628225    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"]
Oct 30 19:42:15.633381 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:15.633351    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-4fg4b"]
Oct 30 19:42:16.187798 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:42:16.187755    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:42:17.542086 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:17.542045    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e20ab810-41dd-465a-8883-08f583b528ee" path="/var/lib/kubelet/pods/e20ab810-41dd-465a-8883-08f583b528ee/volumes"
Oct 30 19:42:30.971353 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.971273    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"]
Oct 30 19:42:30.972003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.971984    5232 cpu_manager.go:414] "containerMap: removing container" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy"
Oct 30 19:42:30.972074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972005    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy"
Oct 30 19:42:30.972074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972028    5232 cpu_manager.go:414] "containerMap: removing container" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy"
Oct 30 19:42:30.972074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972034    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy"
Oct 30 19:42:30.972074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972045    5232 cpu_manager.go:414] "containerMap: removing container" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy"
Oct 30 19:42:30.972074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972051    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy"
Oct 30 19:42:30.972074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972066    5232 cpu_manager.go:414] "containerMap: removing container" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy"
Oct 30 19:42:30.972074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972071    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy"
Oct 30 19:42:30.972354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972091    5232 cpu_manager.go:414] "containerMap: removing container" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy"
Oct 30 19:42:30.972354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972096    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy"
Oct 30 19:42:30.972354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972184    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="b18f59f9-7bd1-4691-b69e-9ab4e4134f5d" containerName="istio-proxy"
Oct 30 19:42:30.972354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972198    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5c9cad5-cd7c-4c0d-a6c4-9fd55e221ed7" containerName="istio-proxy"
Oct 30 19:42:30.972354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972207    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy"
Oct 30 19:42:30.972354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972213    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="691fea7b-46d8-4d38-878b-14623681a293" containerName="istio-proxy"
Oct 30 19:42:30.972354 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.972220    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="e20ab810-41dd-465a-8883-08f583b528ee" containerName="istio-proxy"
Oct 30 19:42:30.975605 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:30.975588    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.031724 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.031692    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-85f471\"/\"kube-root-ca.crt\""
Oct 30 19:42:31.040618 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.040589    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-85f471\"/\"openshift-service-ca.crt\""
Oct 30 19:42:31.074147 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.074104    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-85f471\"/\"auth-test-model-85f471-kserve-self-signed-certs\""
Oct 30 19:42:31.074300 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.074160    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-85f471\"/\"default-dockercfg-zx9w5\""
Oct 30 19:42:31.077467 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.077445    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"]
Oct 30 19:42:31.151536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.151499    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b72541bf-5382-487a-a562-3015b45f188a-tls-certs\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.151536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.151540    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-tmp-dir\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.151761 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.151628    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t2f2\" (UniqueName: \"kubernetes.io/projected/b72541bf-5382-487a-a562-3015b45f188a-kube-api-access-8t2f2\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.151801 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.151772    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-model-cache\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.151945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.151928    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-dshm\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.151984 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.151956    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-home\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.253666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.253577    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-tmp-dir\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.253666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.253624    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t2f2\" (UniqueName: \"kubernetes.io/projected/b72541bf-5382-487a-a562-3015b45f188a-kube-api-access-8t2f2\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.253886 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.253709    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-model-cache\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.253925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.253879    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-dshm\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.253925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.253915    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-home\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.254027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.253952    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b72541bf-5382-487a-a562-3015b45f188a-tls-certs\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.254105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.254078    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-tmp-dir\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.254173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.254112    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-model-cache\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.254309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.254288    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-home\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.256202 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.256179    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-dshm\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.256420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.256404    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b72541bf-5382-487a-a562-3015b45f188a-tls-certs\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.313119 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.313086    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t2f2\" (UniqueName: \"kubernetes.io/projected/b72541bf-5382-487a-a562-3015b45f188a-kube-api-access-8t2f2\") pod \"auth-test-model-85f471-kserve-6d66996db5-wqndd\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") " pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.588645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.588614    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:31.966536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:31.966446    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"]
Oct 30 19:42:31.970961 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:42:31.970928    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb72541bf_5382_487a_a562_3015b45f188a.slice/crio-668fb2c2dc563bfab4d7ce678825135802cbad3d150f92f006924181ccf0008c WatchSource:0}: Error finding container 668fb2c2dc563bfab4d7ce678825135802cbad3d150f92f006924181ccf0008c: Status 404 returned error can't find the container with id 668fb2c2dc563bfab4d7ce678825135802cbad3d150f92f006924181ccf0008c
Oct 30 19:42:32.198236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:32.198195    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd" event={"ID":"b72541bf-5382-487a-a562-3015b45f188a","Type":"ContainerStarted","Data":"e83df5ef6a0bbcc5b267486c8690c59658bd8664e21c0f6f8928a18db65d9d2d"}
Oct 30 19:42:32.198638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:32.198245    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd" event={"ID":"b72541bf-5382-487a-a562-3015b45f188a","Type":"ContainerStarted","Data":"668fb2c2dc563bfab4d7ce678825135802cbad3d150f92f006924181ccf0008c"}
Oct 30 19:42:32.313848 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:32.313716    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd" podStartSLOduration=2.313702324 podStartE2EDuration="2.313702324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:32.311440289 +0000 UTC m=+3344.895923288" watchObservedRunningTime="2026-10-30 19:42:32.313702324 +0000 UTC m=+3344.898185320"
Oct 30 19:42:32.588995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:32.588963    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:41.589274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:41.589231    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:41.602167 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:41.602140    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:42.281431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:42.281396    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:42:46.188201 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:42:46.188163    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:42:59.778775 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:59.778739    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"]
Oct 30 19:42:59.779315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:59.778988    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy" containerID="cri-o://654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938" gracePeriod=30
Oct 30 19:42:59.825877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:59.825841    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"]
Oct 30 19:42:59.826065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:42:59.826030    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy" containerID="cri-o://f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429" gracePeriod=30
Oct 30 19:43:00.404115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:00.404083    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"]
Oct 30 19:43:00.404330 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:00.404301    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy" containerID="cri-o://d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788" gracePeriod=30
Oct 30 19:43:05.050716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.050632    5232 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:43:05.051109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.050707    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 19:43:05.105056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.105023    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:43:05.134108 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.134082    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:43:05.206676 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.206638    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-data\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.206916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.206739    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-socket\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.206916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.206786    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-podinfo\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.206916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.206846    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-token\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-token\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.206916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.206896    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-data\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.206938    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-envoy\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.206969    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-podinfo\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.206979    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-data" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.207159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207025    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-token\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-token\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207026    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-socket" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.207159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207107    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-socket\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207159 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207158    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-kube-api-access-fpqjd\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-kube-api-access-fpqjd\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207179    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-data" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207194    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-certs\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207242    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-certs\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207301    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-envoy\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207362    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-kube-api-access-qpqp7\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-kube-api-access-qpqp7\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207389    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-socket" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207393    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istiod-ca-cert\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207450    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istiod-ca-cert\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207485    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-credential-socket\") pod \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\" (UID: \"16b2b627-fd1a-4e81-8ee3-de291fb99da2\") "
Oct 30 19:43:05.207561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207528    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-credential-socket\") pod \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\" (UID: \"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3\") "
Oct 30 19:43:05.208125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207690    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istiod-ca-cert" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:05.208125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207967    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-credential-socket" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.208125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.207975    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-certs" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.208125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208024    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istiod-ca-cert" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:05.208347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208171    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-credential-socket" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.208347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208183    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-certs" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.208347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208312    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208335    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208351    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208365    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208378    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208393    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208409    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208422    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208436    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.208568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.208450    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.209911 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.209778    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-kube-api-access-fpqjd" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "kube-api-access-fpqjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:05.209911 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.209888    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-token" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:05.210079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.209936    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-envoy" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.210131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.210077    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-podinfo" pod "16b2b627-fd1a-4e81-8ee3-de291fb99da2" (UID: "16b2b627-fd1a-4e81-8ee3-de291fb99da2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:05.210257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.210238    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-token" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:05.210503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.210488    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-kube-api-access-qpqp7" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "kube-api-access-qpqp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:05.210673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.210650    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-envoy" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.211160 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.211146    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-podinfo" pod "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" (UID: "fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:05.309659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.309563    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.309659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.309595    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpqp7\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-kube-api-access-qpqp7\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.309659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.309611    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.309659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.309628    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.309659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.309641    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.309659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.309657    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.309993 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.309669    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.309993 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.309683    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpqjd\" (UniqueName: \"kubernetes.io/projected/16b2b627-fd1a-4e81-8ee3-de291fb99da2-kube-api-access-fpqjd\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.429623 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.429578    5232 generic.go:356] "Generic (PLEG): container finished" podID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerID="f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429" exitCode=0
Oct 30 19:43:05.429768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.429681    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" event={"ID":"16b2b627-fd1a-4e81-8ee3-de291fb99da2","Type":"ContainerDied","Data":"f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429"}
Oct 30 19:43:05.429768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.429696    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"
Oct 30 19:43:05.429768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.429745    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7" event={"ID":"16b2b627-fd1a-4e81-8ee3-de291fb99da2","Type":"ContainerDied","Data":"682e3c4982be4dc9d7bd47a372696ac972bb789a1217ea478ff80ab438433ecb"}
Oct 30 19:43:05.429970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.429775    5232 scope.go:122] "RemoveContainer" containerID="f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429"
Oct 30 19:43:05.431626 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.431599    5232 generic.go:356] "Generic (PLEG): container finished" podID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerID="654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938" exitCode=0
Oct 30 19:43:05.431754 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.431641    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"
Oct 30 19:43:05.431754 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.431648    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" event={"ID":"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3","Type":"ContainerDied","Data":"654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938"}
Oct 30 19:43:05.431754 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.431685    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp" event={"ID":"fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3","Type":"ContainerDied","Data":"9f9906f9352fc5058fea296856fec7fd216352dcff7104a363de26f3216dff6b"}
Oct 30 19:43:05.517404 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.517381    5232 scope.go:122] "RemoveContainer" containerID="f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429"
Oct 30 19:43:05.517792 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:43:05.517765    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429\": container with ID starting with f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429 not found: ID does not exist" containerID="f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429"
Oct 30 19:43:05.517882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.517825    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429"} err="failed to get container status \"f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429\": rpc error: code = NotFound desc = could not find container \"f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429\": container with ID starting with f2f0ad749453e1a083308dad71d4163faf9055e399e528026472d3feb43ee429 not found: ID does not exist"
Oct 30 19:43:05.517882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.517855    5232 scope.go:122] "RemoveContainer" containerID="654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938"
Oct 30 19:43:05.527349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.527328    5232 scope.go:122] "RemoveContainer" containerID="654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938"
Oct 30 19:43:05.527642 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:43:05.527621    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938\": container with ID starting with 654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938 not found: ID does not exist" containerID="654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938"
Oct 30 19:43:05.527716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.527653    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938"} err="failed to get container status \"654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938\": rpc error: code = NotFound desc = could not find container \"654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938\": container with ID starting with 654c29a2af311931e8101e0c6790d67497297cd4ccd7bfcef87d85483ce03938 not found: ID does not exist"
Oct 30 19:43:05.594431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.593338    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"]
Oct 30 19:43:05.604276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.597787    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-m4db7"]
Oct 30 19:43:05.661552 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.661515    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:43:05.700193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.700158    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"]
Oct 30 19:43:05.706972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.706942    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-k8bzp"]
Oct 30 19:43:05.715236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715212    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-data\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715346 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715279    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-credential-socket\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715346 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715327    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-token\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-token\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715487 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715470    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-kube-api-access-8gcgj\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-kube-api-access-8gcgj\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715487 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715476    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-data" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.715599 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715531    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8375f142-2244-4664-9dc2-f9ac7a1dd151-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8375f142-2244-4664-9dc2-f9ac7a1dd151-istiod-ca-cert\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715599 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715562    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-certs\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715599 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715588    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-credential-socket" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.715759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715639    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-envoy\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715677    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-socket\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715737    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-podinfo\") pod \"8375f142-2244-4664-9dc2-f9ac7a1dd151\" (UID: \"8375f142-2244-4664-9dc2-f9ac7a1dd151\") "
Oct 30 19:43:05.715946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715891    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-certs" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.716056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.715959    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8375f142-2244-4664-9dc2-f9ac7a1dd151-istiod-ca-cert" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:05.716056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.716030    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-socket" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.716299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.716274    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.716299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.716301    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.716499 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.716314    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.716499 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.716328    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8375f142-2244-4664-9dc2-f9ac7a1dd151-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.716499 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.716342    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.717660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.717637    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-token" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:05.717914 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.717895    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-podinfo" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:05.717914 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.717901    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-kube-api-access-8gcgj" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "kube-api-access-8gcgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:05.718014 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.717913    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-envoy" pod "8375f142-2244-4664-9dc2-f9ac7a1dd151" (UID: "8375f142-2244-4664-9dc2-f9ac7a1dd151"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.819665 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.819630    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gcgj\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-kube-api-access-8gcgj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.819665 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.819664    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.819947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.819676    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.819947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:05.819685    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8375f142-2244-4664-9dc2-f9ac7a1dd151-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:06.327763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.327726    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"]
Oct 30 19:43:06.328497 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328474    5232 cpu_manager.go:414] "containerMap: removing container" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy"
Oct 30 19:43:06.328590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328499    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy"
Oct 30 19:43:06.328590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328553    5232 cpu_manager.go:414] "containerMap: removing container" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy"
Oct 30 19:43:06.328590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328561    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy"
Oct 30 19:43:06.328590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328587    5232 cpu_manager.go:414] "containerMap: removing container" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy"
Oct 30 19:43:06.328800 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328596    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy"
Oct 30 19:43:06.328800 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328726    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" containerName="istio-proxy"
Oct 30 19:43:06.328800 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328748    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerName="istio-proxy"
Oct 30 19:43:06.328800 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.328760    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" containerName="istio-proxy"
Oct 30 19:43:06.332356 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.332333    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.383403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.383369    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-fe7634\"/\"kube-root-ca.crt\""
Oct 30 19:43:06.383620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.383466    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-fe7634\"/\"default-dockercfg-vfr8j\""
Oct 30 19:43:06.383620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.383521    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-fe7634\"/\"openshift-service-ca.crt\""
Oct 30 19:43:06.383620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.383557    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-fe7634\"/\"auth-test-model-fe7634-kserve-self-signed-certs\""
Oct 30 19:43:06.389846 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.389805    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"]
Oct 30 19:43:06.425782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.425749    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tls-certs\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.425974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.425808    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tmp-dir\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.425974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.425935    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-home\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.425974 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.425966    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-model-cache\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.426096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.426065    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-dshm\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.426138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.426122    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djd6s\" (UniqueName: \"kubernetes.io/projected/19d50bbe-1db0-41c2-8642-6618d5f1bff7-kube-api-access-djd6s\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.439724 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.439690    5232 generic.go:356] "Generic (PLEG): container finished" podID="8375f142-2244-4664-9dc2-f9ac7a1dd151" containerID="d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788" exitCode=0
Oct 30 19:43:06.439921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.439741    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"
Oct 30 19:43:06.439921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.439780    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" event={"ID":"8375f142-2244-4664-9dc2-f9ac7a1dd151","Type":"ContainerDied","Data":"d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788"}
Oct 30 19:43:06.439921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.439861    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc" event={"ID":"8375f142-2244-4664-9dc2-f9ac7a1dd151","Type":"ContainerDied","Data":"86a286fa6b8c0ccba6425474a782e6e9badc0795e8da2d814434dad70678cd82"}
Oct 30 19:43:06.439921 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.439892    5232 scope.go:122] "RemoveContainer" containerID="d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788"
Oct 30 19:43:06.452258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.452238    5232 scope.go:122] "RemoveContainer" containerID="d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788"
Oct 30 19:43:06.452590 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:43:06.452567    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788\": container with ID starting with d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788 not found: ID does not exist" containerID="d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788"
Oct 30 19:43:06.452644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.452599    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788"} err="failed to get container status \"d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788\": rpc error: code = NotFound desc = could not find container \"d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788\": container with ID starting with d37002999f4781e9bd28f964c742307223dc7a74d331b2e24d52c12d029ed788 not found: ID does not exist"
Oct 30 19:43:06.527462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.527413    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djd6s\" (UniqueName: \"kubernetes.io/projected/19d50bbe-1db0-41c2-8642-6618d5f1bff7-kube-api-access-djd6s\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.527668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.527525    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tls-certs\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.527668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.527568    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tmp-dir\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.527668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.527597    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-home\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.527668 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.527617    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-model-cache\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.527937 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.527674    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-dshm\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.528027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.528001    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-home\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.528093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.528014    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tmp-dir\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.528131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.528086    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-model-cache\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.529990 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.529968    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-dshm\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.530228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.530209    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tls-certs\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.586517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.586488    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djd6s\" (UniqueName: \"kubernetes.io/projected/19d50bbe-1db0-41c2-8642-6618d5f1bff7-kube-api-access-djd6s\") pod \"auth-test-model-fe7634-kserve-78646465d5-kcbtl\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") " pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:06.603447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.603415    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"]
Oct 30 19:43:06.610188 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.610154    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-n7tjc"]
Oct 30 19:43:06.644476 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:06.644443    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:07.069602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:07.069289    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"]
Oct 30 19:43:07.073417 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:43:07.073393    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19d50bbe_1db0_41c2_8642_6618d5f1bff7.slice/crio-731265f0e0c9ab68fb03a7e626068571167eca11719ac0b1e81ca8eef0e7bcff WatchSource:0}: Error finding container 731265f0e0c9ab68fb03a7e626068571167eca11719ac0b1e81ca8eef0e7bcff: Status 404 returned error can't find the container with id 731265f0e0c9ab68fb03a7e626068571167eca11719ac0b1e81ca8eef0e7bcff
Oct 30 19:43:07.451755 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:07.451715    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl" event={"ID":"19d50bbe-1db0-41c2-8642-6618d5f1bff7","Type":"ContainerStarted","Data":"5f849bb362a81dfb59444c279ec78fd68ddb84fa097a2efc142e5b78885453a5"}
Oct 30 19:43:07.452185 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:07.451762    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl" event={"ID":"19d50bbe-1db0-41c2-8642-6618d5f1bff7","Type":"ContainerStarted","Data":"731265f0e0c9ab68fb03a7e626068571167eca11719ac0b1e81ca8eef0e7bcff"}
Oct 30 19:43:07.540627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:07.540594    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16b2b627-fd1a-4e81-8ee3-de291fb99da2" path="/var/lib/kubelet/pods/16b2b627-fd1a-4e81-8ee3-de291fb99da2/volumes"
Oct 30 19:43:07.541085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:07.541071    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8375f142-2244-4664-9dc2-f9ac7a1dd151" path="/var/lib/kubelet/pods/8375f142-2244-4664-9dc2-f9ac7a1dd151/volumes"
Oct 30 19:43:07.541480 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:07.541468    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3" path="/var/lib/kubelet/pods/fdf9c17c-6298-4bc1-a98f-4d4c6e67ced3/volumes"
Oct 30 19:43:07.567868 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:07.567799    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl" podStartSLOduration=1.5677860350000001 podStartE2EDuration="1.567786035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:07.566092126 +0000 UTC m=+3380.150575124" watchObservedRunningTime="2026-10-30 19:43:07.567786035 +0000 UTC m=+3380.152269032"
Oct 30 19:43:07.645003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:07.644969    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:16.187779 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:43:16.187735    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:43:16.644872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:16.644829    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:16.657983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:16.657950    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:17.541960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:17.541931    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:43:30.794148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.794107    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"]
Oct 30 19:43:30.798468 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.798446    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:30.897044 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.896990    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w0-d6f58f\"/\"auth-test-model-d6f58f-kserve-self-signed-certs\""
Oct 30 19:43:30.897044 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.897024    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w0-d6f58f\"/\"openshift-service-ca.crt\""
Oct 30 19:43:30.897044 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.897032    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w0-d6f58f\"/\"kube-root-ca.crt\""
Oct 30 19:43:30.897353 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.896990    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w0-d6f58f\"/\"default-dockercfg-rkjpd\""
Oct 30 19:43:30.900123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.900102    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"]
Oct 30 19:43:30.921912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.921889    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2lkg\" (UniqueName: \"kubernetes.io/projected/f6aa2ad7-53f3-43cc-bddf-9021e514b703-kube-api-access-r2lkg\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:30.922002 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.921956    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tls-certs\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:30.922080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.922059    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tmp-dir\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:30.922143 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.922128    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-model-cache\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:30.922245 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.922229    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-home\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:30.922282 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:30.922254    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-dshm\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.023879 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.023847    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tmp-dir\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.024061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.023909    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-model-cache\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.024061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.023988    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-home\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.024061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.024010    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-dshm\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.024061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.024052    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2lkg\" (UniqueName: \"kubernetes.io/projected/f6aa2ad7-53f3-43cc-bddf-9021e514b703-kube-api-access-r2lkg\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.024299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.024135    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tls-certs\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.024299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.024287    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tmp-dir\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.024595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.024570    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-model-cache\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.024727 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.024620    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-home\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.026375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.026357    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-dshm\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.026561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.026544    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tls-certs\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.085441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.085372    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2lkg\" (UniqueName: \"kubernetes.io/projected/f6aa2ad7-53f3-43cc-bddf-9021e514b703-kube-api-access-r2lkg\") pod \"auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") " pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.110236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.110209    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:31.489419 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.489382    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"]
Oct 30 19:43:31.493724 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:43:31.493696    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6aa2ad7_53f3_43cc_bddf_9021e514b703.slice/crio-32e63fa31f8df729bc945a2d4dae8d477d2078759068aa80bf32a70a92b9f38b WatchSource:0}: Error finding container 32e63fa31f8df729bc945a2d4dae8d477d2078759068aa80bf32a70a92b9f38b: Status 404 returned error can't find the container with id 32e63fa31f8df729bc945a2d4dae8d477d2078759068aa80bf32a70a92b9f38b
Oct 30 19:43:31.618777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.618736    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p" event={"ID":"f6aa2ad7-53f3-43cc-bddf-9021e514b703","Type":"ContainerStarted","Data":"1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de"}
Oct 30 19:43:31.618951 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.618788    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p" event={"ID":"f6aa2ad7-53f3-43cc-bddf-9021e514b703","Type":"ContainerStarted","Data":"32e63fa31f8df729bc945a2d4dae8d477d2078759068aa80bf32a70a92b9f38b"}
Oct 30 19:43:31.733250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:31.733197    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p" podStartSLOduration=1.733184068 podStartE2EDuration="1.733184068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:31.731147029 +0000 UTC m=+3404.315630026" watchObservedRunningTime="2026-10-30 19:43:31.733184068 +0000 UTC m=+3404.317667065"
Oct 30 19:43:32.110429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:32.110389    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:41.110603 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:41.110563    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:41.123103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:41.123079    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:41.705877 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:43:41.705843    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:43:46.209675 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:43:46.209631    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:44:16.188704 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:44:16.188665    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:44:46.187018 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:44:46.186982    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:44:58.627822 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:44:58.627771    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"]
Oct 30 19:45:00.261542 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.261509    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"]
Oct 30 19:45:00.265559 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.265539    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"]
Oct 30 19:45:00.265681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.265662    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:00.269422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.269401    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:00.287932 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.287907    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"]
Oct 30 19:45:00.291706 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.291686    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:00.308923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.308897    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"]
Oct 30 19:45:00.312561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.312543    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:00.322415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.322384    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1be50a40-d610-4c64-90f6-408f40c81b79-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b\" (UID: \"1be50a40-d610-4c64-90f6-408f40c81b79\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:00.322705 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.322682    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55cjq\" (UniqueName: \"kubernetes.io/projected/ed017d8a-96be-4e2e-951c-5a23391f1cac-kube-api-access-55cjq\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h\" (UID: \"ed017d8a-96be-4e2e-951c-5a23391f1cac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:00.322967 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.322941    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdb9572-81ee-49b7-aded-db90d1cf5d02-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2\" (UID: \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:00.324197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.323007    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj5vp\" (UniqueName: \"kubernetes.io/projected/1be50a40-d610-4c64-90f6-408f40c81b79-kube-api-access-pj5vp\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b\" (UID: \"1be50a40-d610-4c64-90f6-408f40c81b79\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:00.324197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.323072    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm7f6\" (UniqueName: \"kubernetes.io/projected/5fdb9572-81ee-49b7-aded-db90d1cf5d02-kube-api-access-vm7f6\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2\" (UID: \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:00.324197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.323108    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed017d8a-96be-4e2e-951c-5a23391f1cac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h\" (UID: \"ed017d8a-96be-4e2e-951c-5a23391f1cac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:00.340429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.340399    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"]
Oct 30 19:45:00.347327 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.347304    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"]
Oct 30 19:45:00.353647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.353616    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 19:45:00.400995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.400947    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"]
Oct 30 19:45:00.409688 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.409657    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"]
Oct 30 19:45:00.419652 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.419620    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"]
Oct 30 19:45:00.424157 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424136    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.424313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424292    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdb9572-81ee-49b7-aded-db90d1cf5d02-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2\" (UID: \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:00.424385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424335    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzrh6\" (UniqueName: \"kubernetes.io/projected/9fb9881b-1751-4e74-a32d-b0df066b0915-kube-api-access-gzrh6\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt\" (UID: \"9fb9881b-1751-4e74-a32d-b0df066b0915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:00.424385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424360    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj5vp\" (UniqueName: \"kubernetes.io/projected/1be50a40-d610-4c64-90f6-408f40c81b79-kube-api-access-pj5vp\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b\" (UID: \"1be50a40-d610-4c64-90f6-408f40c81b79\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:00.424496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424462    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm7f6\" (UniqueName: \"kubernetes.io/projected/5fdb9572-81ee-49b7-aded-db90d1cf5d02-kube-api-access-vm7f6\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2\" (UID: \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:00.424555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424501    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed017d8a-96be-4e2e-951c-5a23391f1cac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h\" (UID: \"ed017d8a-96be-4e2e-951c-5a23391f1cac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:00.424613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424572    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb9881b-1751-4e74-a32d-b0df066b0915-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt\" (UID: \"9fb9881b-1751-4e74-a32d-b0df066b0915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:00.424674 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424655    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1be50a40-d610-4c64-90f6-408f40c81b79-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b\" (UID: \"1be50a40-d610-4c64-90f6-408f40c81b79\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:00.424763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.424744    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55cjq\" (UniqueName: \"kubernetes.io/projected/ed017d8a-96be-4e2e-951c-5a23391f1cac-kube-api-access-55cjq\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h\" (UID: \"ed017d8a-96be-4e2e-951c-5a23391f1cac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:00.425183 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.425161    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdb9572-81ee-49b7-aded-db90d1cf5d02-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2\" (UID: \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:00.425344 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.425327    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed017d8a-96be-4e2e-951c-5a23391f1cac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h\" (UID: \"ed017d8a-96be-4e2e-951c-5a23391f1cac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:00.425488 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.425470    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1be50a40-d610-4c64-90f6-408f40c81b79-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b\" (UID: \"1be50a40-d610-4c64-90f6-408f40c81b79\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:00.474414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.474380    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-f858d4-openshift-default-dockercfg-5bq4k\""
Oct 30 19:45:00.486708 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.486677    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj5vp\" (UniqueName: \"kubernetes.io/projected/1be50a40-d610-4c64-90f6-408f40c81b79-kube-api-access-pj5vp\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b\" (UID: \"1be50a40-d610-4c64-90f6-408f40c81b79\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:00.489581 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.489544    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"]
Oct 30 19:45:00.489716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.489646    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm7f6\" (UniqueName: \"kubernetes.io/projected/5fdb9572-81ee-49b7-aded-db90d1cf5d02-kube-api-access-vm7f6\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2\" (UID: \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:00.491148 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.491112    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55cjq\" (UniqueName: \"kubernetes.io/projected/ed017d8a-96be-4e2e-951c-5a23391f1cac-kube-api-access-55cjq\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h\" (UID: \"ed017d8a-96be-4e2e-951c-5a23391f1cac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:00.526261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526170    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-credential-socket\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.526261 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526226    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-podinfo\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.526476 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526318    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb9881b-1751-4e74-a32d-b0df066b0915-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt\" (UID: \"9fb9881b-1751-4e74-a32d-b0df066b0915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:00.526476 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526372    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-certs\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.526476 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526404    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-data\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.526641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526549    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istiod-ca-cert\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.526641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526581    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-envoy\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.526748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526652    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pvrr\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-kube-api-access-4pvrr\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.526803 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526771    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzrh6\" (UniqueName: \"kubernetes.io/projected/9fb9881b-1751-4e74-a32d-b0df066b0915-kube-api-access-gzrh6\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt\" (UID: \"9fb9881b-1751-4e74-a32d-b0df066b0915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:00.526878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526828    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-token\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.526878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.526872    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-socket\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.527056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.527035    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb9881b-1751-4e74-a32d-b0df066b0915-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt\" (UID: \"9fb9881b-1751-4e74-a32d-b0df066b0915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:00.583209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.583178    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:00.591500 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.591471    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:00.605116 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.605086    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzrh6\" (UniqueName: \"kubernetes.io/projected/9fb9881b-1751-4e74-a32d-b0df066b0915-kube-api-access-gzrh6\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt\" (UID: \"9fb9881b-1751-4e74-a32d-b0df066b0915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:00.605241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.605212    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:00.628747 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.628709    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-token\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.628946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.628772    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-socket\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.628946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.628888    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-credential-socket\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.628946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.628942    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-podinfo\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.629128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.628987    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-certs\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.629128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.629031    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-data\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.629128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.629114    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istiod-ca-cert\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.629290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.629169    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-envoy\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.629290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.629217    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pvrr\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-kube-api-access-4pvrr\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.632056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.630984    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-certs\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.632056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.631238    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-credential-socket\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.632272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.632259    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-data\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.633062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.632897    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istiod-ca-cert\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.635039 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.635013    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-podinfo\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.635388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.635362    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-socket\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.635610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.635590    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:00.639375 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.639348    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-envoy\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.723230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.723194    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-token\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.725632 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.725604    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pvrr\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-kube-api-access-4pvrr\") pod \"e2e-autores-f858d4-openshift-default-8b78cc457-86g9m\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") " pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.743102 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.743072    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:00.996431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:00.996395    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"]
Oct 30 19:45:01.001070 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:45:01.001042    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdb9572_81ee_49b7_aded_db90d1cf5d02.slice/crio-e722f4bb40da6a230a2d28d3957a8d718af63d92cc01e1fd0689fe7eba448a7c WatchSource:0}: Error finding container e722f4bb40da6a230a2d28d3957a8d718af63d92cc01e1fd0689fe7eba448a7c: Status 404 returned error can't find the container with id e722f4bb40da6a230a2d28d3957a8d718af63d92cc01e1fd0689fe7eba448a7c
Oct 30 19:45:01.045421 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.045396    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"]
Oct 30 19:45:01.048957 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:45:01.048931    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be50a40_d610_4c64_90f6_408f40c81b79.slice/crio-a9654b16e4b4ed75d2a0c583865ad0752d55c0122e0c7de130835cf927c7d38f WatchSource:0}: Error finding container a9654b16e4b4ed75d2a0c583865ad0752d55c0122e0c7de130835cf927c7d38f: Status 404 returned error can't find the container with id a9654b16e4b4ed75d2a0c583865ad0752d55c0122e0c7de130835cf927c7d38f
Oct 30 19:45:01.049368 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.049346    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"]
Oct 30 19:45:01.050624 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:45:01.050604    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded017d8a_96be_4e2e_951c_5a23391f1cac.slice/crio-41630e0786edccb8761f82d04071d859e13468bb246ef3c9ac26496e8fda415b WatchSource:0}: Error finding container 41630e0786edccb8761f82d04071d859e13468bb246ef3c9ac26496e8fda415b: Status 404 returned error can't find the container with id 41630e0786edccb8761f82d04071d859e13468bb246ef3c9ac26496e8fda415b
Oct 30 19:45:01.070337 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:45:01.070309    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb9881b_1751_4e74_a32d_b0df066b0915.slice/crio-810452cfaea3ff9151886b10fccbc75f5c8579d28b14dc8bb32b06fb497fba7d WatchSource:0}: Error finding container 810452cfaea3ff9151886b10fccbc75f5c8579d28b14dc8bb32b06fb497fba7d: Status 404 returned error can't find the container with id 810452cfaea3ff9151886b10fccbc75f5c8579d28b14dc8bb32b06fb497fba7d
Oct 30 19:45:01.070449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.070409    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"]
Oct 30 19:45:01.183346 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.183315    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"]
Oct 30 19:45:01.240758 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:45:01.240711    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21b1b6c_bcf5_4787_8c1b_c055df3b6264.slice/crio-2111e37a554741d77ad5dcee7e624aaeab87251d1356983774e256f83b782441 WatchSource:0}: Error finding container 2111e37a554741d77ad5dcee7e624aaeab87251d1356983774e256f83b782441: Status 404 returned error can't find the container with id 2111e37a554741d77ad5dcee7e624aaeab87251d1356983774e256f83b782441
Oct 30 19:45:01.244067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.243529    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:01.244067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.243622    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:01.244067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.243669    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:01.250471 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.250263    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt" event={"ID":"9fb9881b-1751-4e74-a32d-b0df066b0915","Type":"ContainerStarted","Data":"810452cfaea3ff9151886b10fccbc75f5c8579d28b14dc8bb32b06fb497fba7d"}
Oct 30 19:45:01.252021 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.251989    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2" event={"ID":"5fdb9572-81ee-49b7-aded-db90d1cf5d02","Type":"ContainerStarted","Data":"e722f4bb40da6a230a2d28d3957a8d718af63d92cc01e1fd0689fe7eba448a7c"}
Oct 30 19:45:01.258548 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.258514    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" event={"ID":"ed017d8a-96be-4e2e-951c-5a23391f1cac","Type":"ContainerStarted","Data":"41630e0786edccb8761f82d04071d859e13468bb246ef3c9ac26496e8fda415b"}
Oct 30 19:45:01.259949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:01.259922    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" event={"ID":"1be50a40-d610-4c64-90f6-408f40c81b79","Type":"ContainerStarted","Data":"a9654b16e4b4ed75d2a0c583865ad0752d55c0122e0c7de130835cf927c7d38f"}
Oct 30 19:45:02.268765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.268728    5232 generic.go:356] "Generic (PLEG): container finished" podID="1be50a40-d610-4c64-90f6-408f40c81b79" containerID="1aa16d0ad5c1070b56ddba4c208a8fb329b130cc2a9028e6e868310adf60c767" exitCode=60
Oct 30 19:45:02.269248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.268845    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" event={"ID":"1be50a40-d610-4c64-90f6-408f40c81b79","Type":"ContainerDied","Data":"1aa16d0ad5c1070b56ddba4c208a8fb329b130cc2a9028e6e868310adf60c767"}
Oct 30 19:45:02.269248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.269121    5232 scope.go:122] "RemoveContainer" containerID="1aa16d0ad5c1070b56ddba4c208a8fb329b130cc2a9028e6e868310adf60c767"
Oct 30 19:45:02.270782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.270690    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" event={"ID":"d21b1b6c-bcf5-4787-8c1b-c055df3b6264","Type":"ContainerStarted","Data":"e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4"}
Oct 30 19:45:02.270782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.270731    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" event={"ID":"d21b1b6c-bcf5-4787-8c1b-c055df3b6264","Type":"ContainerStarted","Data":"2111e37a554741d77ad5dcee7e624aaeab87251d1356983774e256f83b782441"}
Oct 30 19:45:02.272581 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.272558    5232 generic.go:356] "Generic (PLEG): container finished" podID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerID="8d3560b2f6f1e6808569f4fce1eee3d12ede600ecd4ecab789461883fe66fb5a" exitCode=60
Oct 30 19:45:02.272664 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.272588    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt" event={"ID":"9fb9881b-1751-4e74-a32d-b0df066b0915","Type":"ContainerDied","Data":"8d3560b2f6f1e6808569f4fce1eee3d12ede600ecd4ecab789461883fe66fb5a"}
Oct 30 19:45:02.272906 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.272882    5232 scope.go:122] "RemoveContainer" containerID="8d3560b2f6f1e6808569f4fce1eee3d12ede600ecd4ecab789461883fe66fb5a"
Oct 30 19:45:02.274431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.274408    5232 generic.go:356] "Generic (PLEG): container finished" podID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerID="c4a90780af8a8ddb03e849e1bdad1f37aa11a7077e93acbc15f5776e3fc0de96" exitCode=60
Oct 30 19:45:02.274571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.274488    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2" event={"ID":"5fdb9572-81ee-49b7-aded-db90d1cf5d02","Type":"ContainerDied","Data":"c4a90780af8a8ddb03e849e1bdad1f37aa11a7077e93acbc15f5776e3fc0de96"}
Oct 30 19:45:02.274715 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.274700    5232 scope.go:122] "RemoveContainer" containerID="c4a90780af8a8ddb03e849e1bdad1f37aa11a7077e93acbc15f5776e3fc0de96"
Oct 30 19:45:02.276508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.276481    5232 generic.go:356] "Generic (PLEG): container finished" podID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerID="77170510d8f68259bc47bae8189aa887b0b7283159dd0ad24b2043862bf653d6" exitCode=60
Oct 30 19:45:02.276610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.276548    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" event={"ID":"ed017d8a-96be-4e2e-951c-5a23391f1cac","Type":"ContainerDied","Data":"77170510d8f68259bc47bae8189aa887b0b7283159dd0ad24b2043862bf653d6"}
Oct 30 19:45:02.276806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.276780    5232 scope.go:122] "RemoveContainer" containerID="77170510d8f68259bc47bae8189aa887b0b7283159dd0ad24b2043862bf653d6"
Oct 30 19:45:02.743888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.743860    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:02.750477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.750451    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:02.761493 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:02.761446    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" podStartSLOduration=2.7614298809999998 podStartE2EDuration="2.761429881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:02.758634016 +0000 UTC m=+3495.343117014" watchObservedRunningTime="2026-10-30 19:45:02.761429881 +0000 UTC m=+3495.345912877"
Oct 30 19:45:03.290579 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.290548    5232 generic.go:356] "Generic (PLEG): container finished" podID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerID="e893c73fb003150affdd2b23f843cbfa1fa6c3c1658fef02240fc4a7dc8cf65f" exitCode=60
Oct 30 19:45:03.291019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.290626    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt" event={"ID":"9fb9881b-1751-4e74-a32d-b0df066b0915","Type":"ContainerDied","Data":"e893c73fb003150affdd2b23f843cbfa1fa6c3c1658fef02240fc4a7dc8cf65f"}
Oct 30 19:45:03.291019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.290705    5232 scope.go:122] "RemoveContainer" containerID="8d3560b2f6f1e6808569f4fce1eee3d12ede600ecd4ecab789461883fe66fb5a"
Oct 30 19:45:03.291019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.291017    5232 scope.go:122] "RemoveContainer" containerID="e893c73fb003150affdd2b23f843cbfa1fa6c3c1658fef02240fc4a7dc8cf65f"
Oct 30 19:45:03.291353 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:03.291334    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt_odh-ai-gateway-infra(9fb9881b-1751-4e74-a32d-b0df066b0915)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915"
Oct 30 19:45:03.292593 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.292574    5232 generic.go:356] "Generic (PLEG): container finished" podID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerID="44e8553f02811b74d22c2109bfe7cf6ea9031948635e5e3ca75a9cc969508f5d" exitCode=60
Oct 30 19:45:03.292678 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.292643    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2" event={"ID":"5fdb9572-81ee-49b7-aded-db90d1cf5d02","Type":"ContainerDied","Data":"44e8553f02811b74d22c2109bfe7cf6ea9031948635e5e3ca75a9cc969508f5d"}
Oct 30 19:45:03.292936 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.292919    5232 scope.go:122] "RemoveContainer" containerID="44e8553f02811b74d22c2109bfe7cf6ea9031948635e5e3ca75a9cc969508f5d"
Oct 30 19:45:03.293219 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:03.293198    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2_odh-ai-gateway-infra(5fdb9572-81ee-49b7-aded-db90d1cf5d02)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02"
Oct 30 19:45:03.294433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.294408    5232 generic.go:356] "Generic (PLEG): container finished" podID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerID="51484d8082836d2345646ab17d9e00e44f454c651d7fa3b0220308b790696227" exitCode=60
Oct 30 19:45:03.294510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.294490    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" event={"ID":"ed017d8a-96be-4e2e-951c-5a23391f1cac","Type":"ContainerDied","Data":"51484d8082836d2345646ab17d9e00e44f454c651d7fa3b0220308b790696227"}
Oct 30 19:45:03.294720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.294703    5232 scope.go:122] "RemoveContainer" containerID="51484d8082836d2345646ab17d9e00e44f454c651d7fa3b0220308b790696227"
Oct 30 19:45:03.295018 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:03.294992    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h_odh-ai-gateway-infra(ed017d8a-96be-4e2e-951c-5a23391f1cac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac"
Oct 30 19:45:03.296299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.296278    5232 generic.go:356] "Generic (PLEG): container finished" podID="1be50a40-d610-4c64-90f6-408f40c81b79" containerID="285c3551d2805d4cc7c20394e91640bf91bd4cc2396b20363db3c065742e6b35" exitCode=60
Oct 30 19:45:03.296407 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.296350    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" event={"ID":"1be50a40-d610-4c64-90f6-408f40c81b79","Type":"ContainerDied","Data":"285c3551d2805d4cc7c20394e91640bf91bd4cc2396b20363db3c065742e6b35"}
Oct 30 19:45:03.296526 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.296512    5232 scope.go:122] "RemoveContainer" containerID="285c3551d2805d4cc7c20394e91640bf91bd4cc2396b20363db3c065742e6b35"
Oct 30 19:45:03.296715 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:03.296696    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b_odh-ai-gateway-infra(1be50a40-d610-4c64-90f6-408f40c81b79)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" podUID="1be50a40-d610-4c64-90f6-408f40c81b79"
Oct 30 19:45:03.296763 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.296704    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:03.297758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.297740    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:45:03.369620 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.369600    5232 scope.go:122] "RemoveContainer" containerID="c4a90780af8a8ddb03e849e1bdad1f37aa11a7077e93acbc15f5776e3fc0de96"
Oct 30 19:45:03.383445 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.383420    5232 scope.go:122] "RemoveContainer" containerID="77170510d8f68259bc47bae8189aa887b0b7283159dd0ad24b2043862bf653d6"
Oct 30 19:45:03.399178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:03.399157    5232 scope.go:122] "RemoveContainer" containerID="1aa16d0ad5c1070b56ddba4c208a8fb329b130cc2a9028e6e868310adf60c767"
Oct 30 19:45:04.305205 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:04.305177    5232 scope.go:122] "RemoveContainer" containerID="e893c73fb003150affdd2b23f843cbfa1fa6c3c1658fef02240fc4a7dc8cf65f"
Oct 30 19:45:04.305671 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:04.305506    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt_odh-ai-gateway-infra(9fb9881b-1751-4e74-a32d-b0df066b0915)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915"
Oct 30 19:45:04.306888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:04.306869    5232 scope.go:122] "RemoveContainer" containerID="44e8553f02811b74d22c2109bfe7cf6ea9031948635e5e3ca75a9cc969508f5d"
Oct 30 19:45:04.307180 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:04.307158    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2_odh-ai-gateway-infra(5fdb9572-81ee-49b7-aded-db90d1cf5d02)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02"
Oct 30 19:45:04.308538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:04.308520    5232 scope.go:122] "RemoveContainer" containerID="51484d8082836d2345646ab17d9e00e44f454c651d7fa3b0220308b790696227"
Oct 30 19:45:04.308754 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:04.308733    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h_odh-ai-gateway-infra(ed017d8a-96be-4e2e-951c-5a23391f1cac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac"
Oct 30 19:45:04.310083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:04.310067    5232 scope.go:122] "RemoveContainer" containerID="285c3551d2805d4cc7c20394e91640bf91bd4cc2396b20363db3c065742e6b35"
Oct 30 19:45:04.310273 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:04.310256    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b_odh-ai-gateway-infra(1be50a40-d610-4c64-90f6-408f40c81b79)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" podUID="1be50a40-d610-4c64-90f6-408f40c81b79"
Oct 30 19:45:13.533760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:13.533666    5232 scope.go:122] "RemoveContainer" containerID="44e8553f02811b74d22c2109bfe7cf6ea9031948635e5e3ca75a9cc969508f5d"
Oct 30 19:45:13.653710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:13.653660    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p" podUID="f6aa2ad7-53f3-43cc-bddf-9021e514b703" containerName="main" containerID="cri-o://1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de" gracePeriod=45
Oct 30 19:45:13.695647 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:13.695612    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p" podUID="f6aa2ad7-53f3-43cc-bddf-9021e514b703" containerName="main" probeResult="failure" output="Get \"https://10.133.0.161:8000/ready\": dial tcp 10.133.0.161:8000: connect: connection refused"
Oct 30 19:45:13.943722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:13.943690    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:45:14.113019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.112984    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6aa2ad7-53f3-43cc-bddf-9021e514b703-kube-api-access-r2lkg\" (UniqueName: \"kubernetes.io/projected/f6aa2ad7-53f3-43cc-bddf-9021e514b703-kube-api-access-r2lkg\") pod \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") "
Oct 30 19:45:14.113197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113061    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tmp-dir\") pod \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") "
Oct 30 19:45:14.113292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113194    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tls-certs\" (UniqueName: \"kubernetes.io/secret/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tls-certs\") pod \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") "
Oct 30 19:45:14.113292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113236    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-dshm\") pod \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") "
Oct 30 19:45:14.113402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113295    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-model-cache\") pod \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") "
Oct 30 19:45:14.113402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113380    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tmp-dir" pod "f6aa2ad7-53f3-43cc-bddf-9021e514b703" (UID: "f6aa2ad7-53f3-43cc-bddf-9021e514b703"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:14.113508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113393    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-home\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-home\") pod \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\" (UID: \"f6aa2ad7-53f3-43cc-bddf-9021e514b703\") "
Oct 30 19:45:14.113614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113590    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-home" pod "f6aa2ad7-53f3-43cc-bddf-9021e514b703" (UID: "f6aa2ad7-53f3-43cc-bddf-9021e514b703"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:14.113666 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113609    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-model-cache" pod "f6aa2ad7-53f3-43cc-bddf-9021e514b703" (UID: "f6aa2ad7-53f3-43cc-bddf-9021e514b703"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:14.113993 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113973    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:14.114086 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.113997    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:14.114086 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.114011    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:14.115527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.115502    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-dshm" pod "f6aa2ad7-53f3-43cc-bddf-9021e514b703" (UID: "f6aa2ad7-53f3-43cc-bddf-9021e514b703"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:14.115617 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.115587    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tls-certs" pod "f6aa2ad7-53f3-43cc-bddf-9021e514b703" (UID: "f6aa2ad7-53f3-43cc-bddf-9021e514b703"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:14.115617 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.115591    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6aa2ad7-53f3-43cc-bddf-9021e514b703-kube-api-access-r2lkg" pod "f6aa2ad7-53f3-43cc-bddf-9021e514b703" (UID: "f6aa2ad7-53f3-43cc-bddf-9021e514b703"). InnerVolumeSpecName "kube-api-access-r2lkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:14.215733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.215702    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6aa2ad7-53f3-43cc-bddf-9021e514b703-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:14.215733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.215729    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6aa2ad7-53f3-43cc-bddf-9021e514b703-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:14.215733 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.215739    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2lkg\" (UniqueName: \"kubernetes.io/projected/f6aa2ad7-53f3-43cc-bddf-9021e514b703-kube-api-access-r2lkg\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:14.379239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.379166    5232 generic.go:356] "Generic (PLEG): container finished" podID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerID="aa256abec4aa6da39b671b9b58ad7df0a2701e604b39f873681f694511c21f3f" exitCode=60
Oct 30 19:45:14.379449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.379235    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2" event={"ID":"5fdb9572-81ee-49b7-aded-db90d1cf5d02","Type":"ContainerDied","Data":"aa256abec4aa6da39b671b9b58ad7df0a2701e604b39f873681f694511c21f3f"}
Oct 30 19:45:14.379449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.379293    5232 scope.go:122] "RemoveContainer" containerID="44e8553f02811b74d22c2109bfe7cf6ea9031948635e5e3ca75a9cc969508f5d"
Oct 30 19:45:14.379710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.379692    5232 scope.go:122] "RemoveContainer" containerID="aa256abec4aa6da39b671b9b58ad7df0a2701e604b39f873681f694511c21f3f"
Oct 30 19:45:14.380140 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:14.380112    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2_odh-ai-gateway-infra(5fdb9572-81ee-49b7-aded-db90d1cf5d02)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02"
Oct 30 19:45:14.380878 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.380858    5232 generic.go:356] "Generic (PLEG): container finished" podID="f6aa2ad7-53f3-43cc-bddf-9021e514b703" containerID="1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de" exitCode=0
Oct 30 19:45:14.380966 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.380903    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"
Oct 30 19:45:14.380966 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.380938    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p" event={"ID":"f6aa2ad7-53f3-43cc-bddf-9021e514b703","Type":"ContainerDied","Data":"1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de"}
Oct 30 19:45:14.381081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.380972    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p" event={"ID":"f6aa2ad7-53f3-43cc-bddf-9021e514b703","Type":"ContainerDied","Data":"32e63fa31f8df729bc945a2d4dae8d477d2078759068aa80bf32a70a92b9f38b"}
Oct 30 19:45:14.395409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.395391    5232 scope.go:122] "RemoveContainer" containerID="1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de"
Oct 30 19:45:14.404637 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.404607    5232 scope.go:122] "RemoveContainer" containerID="1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de"
Oct 30 19:45:14.404928 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:14.404907    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de\": container with ID starting with 1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de not found: ID does not exist" containerID="1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de"
Oct 30 19:45:14.405006 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.404942    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de"} err="failed to get container status \"1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de\": rpc error: code = NotFound desc = could not find container \"1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de\": container with ID starting with 1081fc262ef8d9f0ebdfa8e81ffdbb14e354e5a2354257a8da6dec8ad15600de not found: ID does not exist"
Oct 30 19:45:14.533663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.533635    5232 scope.go:122] "RemoveContainer" containerID="51484d8082836d2345646ab17d9e00e44f454c651d7fa3b0220308b790696227"
Oct 30 19:45:14.603199 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.603164    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"]
Oct 30 19:45:14.606205 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:14.606176    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w0-d6f58f/auth-test-model-d6f58f-kserve-9f64b5b97-jsl2p"]
Oct 30 19:45:15.390298 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.390259    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" event={"ID":"ed017d8a-96be-4e2e-951c-5a23391f1cac","Type":"ContainerStarted","Data":"aea6364299fe63579f85452cbb58c723cf9cbe640d88a5d1f7faf661d3a9c15c"}
Oct 30 19:45:15.513780 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.513743    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"]
Oct 30 19:45:15.516128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.516080    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" podStartSLOduration=15.516068397 podStartE2EDuration="15.516068397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:15.509883111 +0000 UTC m=+3508.094366111" watchObservedRunningTime="2026-10-30 19:45:15.516068397 +0000 UTC m=+3508.100551394"
Oct 30 19:45:15.533146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.533115    5232 scope.go:122] "RemoveContainer" containerID="285c3551d2805d4cc7c20394e91640bf91bd4cc2396b20363db3c065742e6b35"
Oct 30 19:45:15.543256 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.543226    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6aa2ad7-53f3-43cc-bddf-9021e514b703" path="/var/lib/kubelet/pods/f6aa2ad7-53f3-43cc-bddf-9021e514b703/volumes"
Oct 30 19:45:15.675101 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.675069    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:15.835864 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.835797    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fdb9572-81ee-49b7-aded-db90d1cf5d02-kube-api-access-vm7f6\" (UniqueName: \"kubernetes.io/projected/5fdb9572-81ee-49b7-aded-db90d1cf5d02-kube-api-access-vm7f6\") pod \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\" (UID: \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\") "
Oct 30 19:45:15.836056 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.835905    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fdb9572-81ee-49b7-aded-db90d1cf5d02-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdb9572-81ee-49b7-aded-db90d1cf5d02-maas-api-service-ca\") pod \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\" (UID: \"5fdb9572-81ee-49b7-aded-db90d1cf5d02\") "
Oct 30 19:45:15.836340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.836315    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fdb9572-81ee-49b7-aded-db90d1cf5d02-maas-api-service-ca" pod "5fdb9572-81ee-49b7-aded-db90d1cf5d02" (UID: "5fdb9572-81ee-49b7-aded-db90d1cf5d02"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:15.836528 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.836511    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdb9572-81ee-49b7-aded-db90d1cf5d02-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:15.838353 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.838328    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fdb9572-81ee-49b7-aded-db90d1cf5d02-kube-api-access-vm7f6" pod "5fdb9572-81ee-49b7-aded-db90d1cf5d02" (UID: "5fdb9572-81ee-49b7-aded-db90d1cf5d02"). InnerVolumeSpecName "kube-api-access-vm7f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:15.937888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:15.937788    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm7f6\" (UniqueName: \"kubernetes.io/projected/5fdb9572-81ee-49b7-aded-db90d1cf5d02-kube-api-access-vm7f6\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:16.191030 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:16.190942    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:45:16.401471 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.401115    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"
Oct 30 19:45:16.401471 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.401128    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2" event={"ID":"5fdb9572-81ee-49b7-aded-db90d1cf5d02","Type":"ContainerDied","Data":"e722f4bb40da6a230a2d28d3957a8d718af63d92cc01e1fd0689fe7eba448a7c"}
Oct 30 19:45:16.401471 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.401197    5232 scope.go:122] "RemoveContainer" containerID="aa256abec4aa6da39b671b9b58ad7df0a2701e604b39f873681f694511c21f3f"
Oct 30 19:45:16.404264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.404236    5232 generic.go:356] "Generic (PLEG): container finished" podID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerID="aea6364299fe63579f85452cbb58c723cf9cbe640d88a5d1f7faf661d3a9c15c" exitCode=60
Oct 30 19:45:16.404368 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.404315    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" event={"ID":"ed017d8a-96be-4e2e-951c-5a23391f1cac","Type":"ContainerDied","Data":"aea6364299fe63579f85452cbb58c723cf9cbe640d88a5d1f7faf661d3a9c15c"}
Oct 30 19:45:16.404792 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.404768    5232 scope.go:122] "RemoveContainer" containerID="aea6364299fe63579f85452cbb58c723cf9cbe640d88a5d1f7faf661d3a9c15c"
Oct 30 19:45:16.405212 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:16.405075    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h_odh-ai-gateway-infra(ed017d8a-96be-4e2e-951c-5a23391f1cac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac"
Oct 30 19:45:16.406485 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.406462    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" event={"ID":"1be50a40-d610-4c64-90f6-408f40c81b79","Type":"ContainerStarted","Data":"d9ed956f6689b49c9352f7bb6a652c020f28a7281b22ea4f1e8e9c4c6177b9dd"}
Oct 30 19:45:16.414300 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.413411    5232 scope.go:122] "RemoveContainer" containerID="51484d8082836d2345646ab17d9e00e44f454c651d7fa3b0220308b790696227"
Oct 30 19:45:16.524271 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.524197    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"]
Oct 30 19:45:16.533641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.533612    5232 scope.go:122] "RemoveContainer" containerID="e893c73fb003150affdd2b23f843cbfa1fa6c3c1658fef02240fc4a7dc8cf65f"
Oct 30 19:45:16.566433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.566381    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" podStartSLOduration=16.56636764 podStartE2EDuration="16.56636764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:16.564364859 +0000 UTC m=+3509.148847856" watchObservedRunningTime="2026-10-30 19:45:16.56636764 +0000 UTC m=+3509.150850636"
Oct 30 19:45:16.674767 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.674718    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"]
Oct 30 19:45:16.677701 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:16.677675    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe7634-29889825-wt8k2"]
Oct 30 19:45:17.415853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.415788    5232 generic.go:356] "Generic (PLEG): container finished" podID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerID="d03ca58e4e6d6ccc1302cdde8ca8db916b09e136d586f252b2faeb9066710360" exitCode=60
Oct 30 19:45:17.416071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.415865    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt" event={"ID":"9fb9881b-1751-4e74-a32d-b0df066b0915","Type":"ContainerDied","Data":"d03ca58e4e6d6ccc1302cdde8ca8db916b09e136d586f252b2faeb9066710360"}
Oct 30 19:45:17.416071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.415934    5232 scope.go:122] "RemoveContainer" containerID="e893c73fb003150affdd2b23f843cbfa1fa6c3c1658fef02240fc4a7dc8cf65f"
Oct 30 19:45:17.416340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.416320    5232 scope.go:122] "RemoveContainer" containerID="d03ca58e4e6d6ccc1302cdde8ca8db916b09e136d586f252b2faeb9066710360"
Oct 30 19:45:17.416754 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:17.416659    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt_odh-ai-gateway-infra(9fb9881b-1751-4e74-a32d-b0df066b0915)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915"
Oct 30 19:45:17.424028 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.423998    5232 generic.go:356] "Generic (PLEG): container finished" podID="1be50a40-d610-4c64-90f6-408f40c81b79" containerID="d9ed956f6689b49c9352f7bb6a652c020f28a7281b22ea4f1e8e9c4c6177b9dd" exitCode=60
Oct 30 19:45:17.424142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.424063    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" event={"ID":"1be50a40-d610-4c64-90f6-408f40c81b79","Type":"ContainerDied","Data":"d9ed956f6689b49c9352f7bb6a652c020f28a7281b22ea4f1e8e9c4c6177b9dd"}
Oct 30 19:45:17.424485 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.424465    5232 scope.go:122] "RemoveContainer" containerID="d9ed956f6689b49c9352f7bb6a652c020f28a7281b22ea4f1e8e9c4c6177b9dd"
Oct 30 19:45:17.424875 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:17.424847    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b_odh-ai-gateway-infra(1be50a40-d610-4c64-90f6-408f40c81b79)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" podUID="1be50a40-d610-4c64-90f6-408f40c81b79"
Oct 30 19:45:17.431045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.431019    5232 scope.go:122] "RemoveContainer" containerID="285c3551d2805d4cc7c20394e91640bf91bd4cc2396b20363db3c065742e6b35"
Oct 30 19:45:17.544167 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.544132    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" path="/var/lib/kubelet/pods/5fdb9572-81ee-49b7-aded-db90d1cf5d02/volumes"
Oct 30 19:45:17.570945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.570923    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:17.578009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.577987    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"]
Oct 30 19:45:17.661010 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.660926    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed017d8a-96be-4e2e-951c-5a23391f1cac-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed017d8a-96be-4e2e-951c-5a23391f1cac-maas-api-service-ca\") pod \"ed017d8a-96be-4e2e-951c-5a23391f1cac\" (UID: \"ed017d8a-96be-4e2e-951c-5a23391f1cac\") "
Oct 30 19:45:17.661189 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.661138    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed017d8a-96be-4e2e-951c-5a23391f1cac-kube-api-access-55cjq\" (UniqueName: \"kubernetes.io/projected/ed017d8a-96be-4e2e-951c-5a23391f1cac-kube-api-access-55cjq\") pod \"ed017d8a-96be-4e2e-951c-5a23391f1cac\" (UID: \"ed017d8a-96be-4e2e-951c-5a23391f1cac\") "
Oct 30 19:45:17.661377 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.661346    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed017d8a-96be-4e2e-951c-5a23391f1cac-maas-api-service-ca" pod "ed017d8a-96be-4e2e-951c-5a23391f1cac" (UID: "ed017d8a-96be-4e2e-951c-5a23391f1cac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:17.661581 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.661564    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed017d8a-96be-4e2e-951c-5a23391f1cac-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:17.663475 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.663442    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed017d8a-96be-4e2e-951c-5a23391f1cac-kube-api-access-55cjq" pod "ed017d8a-96be-4e2e-951c-5a23391f1cac" (UID: "ed017d8a-96be-4e2e-951c-5a23391f1cac"). InnerVolumeSpecName "kube-api-access-55cjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:17.762840 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:17.762771    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55cjq\" (UniqueName: \"kubernetes.io/projected/ed017d8a-96be-4e2e-951c-5a23391f1cac-kube-api-access-55cjq\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.433435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.433402    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"
Oct 30 19:45:18.433435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.433423    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h" event={"ID":"ed017d8a-96be-4e2e-951c-5a23391f1cac","Type":"ContainerDied","Data":"41630e0786edccb8761f82d04071d859e13468bb246ef3c9ac26496e8fda415b"}
Oct 30 19:45:18.433699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.433489    5232 scope.go:122] "RemoveContainer" containerID="aea6364299fe63579f85452cbb58c723cf9cbe640d88a5d1f7faf661d3a9c15c"
Oct 30 19:45:18.562203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.562174    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"]
Oct 30 19:45:18.594050 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.594022    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:18.671708 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.671673    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1be50a40-d610-4c64-90f6-408f40c81b79-kube-api-access-pj5vp\" (UniqueName: \"kubernetes.io/projected/1be50a40-d610-4c64-90f6-408f40c81b79-kube-api-access-pj5vp\") pod \"1be50a40-d610-4c64-90f6-408f40c81b79\" (UID: \"1be50a40-d610-4c64-90f6-408f40c81b79\") "
Oct 30 19:45:18.671929 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.671733    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1be50a40-d610-4c64-90f6-408f40c81b79-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1be50a40-d610-4c64-90f6-408f40c81b79-maas-api-service-ca\") pod \"1be50a40-d610-4c64-90f6-408f40c81b79\" (UID: \"1be50a40-d610-4c64-90f6-408f40c81b79\") "
Oct 30 19:45:18.672125 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.672101    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1be50a40-d610-4c64-90f6-408f40c81b79-maas-api-service-ca" pod "1be50a40-d610-4c64-90f6-408f40c81b79" (UID: "1be50a40-d610-4c64-90f6-408f40c81b79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:18.672366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.672348    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1be50a40-d610-4c64-90f6-408f40c81b79-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.674065 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.674038    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1be50a40-d610-4c64-90f6-408f40c81b79-kube-api-access-pj5vp" pod "1be50a40-d610-4c64-90f6-408f40c81b79" (UID: "1be50a40-d610-4c64-90f6-408f40c81b79"). InnerVolumeSpecName "kube-api-access-pj5vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:18.718315 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.718285    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"]
Oct 30 19:45:18.728673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.728650    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:18.736558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.736025    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-c01685-29889825-gb55h"]
Oct 30 19:45:18.773726 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.773691    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb9881b-1751-4e74-a32d-b0df066b0915-kube-api-access-gzrh6\" (UniqueName: \"kubernetes.io/projected/9fb9881b-1751-4e74-a32d-b0df066b0915-kube-api-access-gzrh6\") pod \"9fb9881b-1751-4e74-a32d-b0df066b0915\" (UID: \"9fb9881b-1751-4e74-a32d-b0df066b0915\") "
Oct 30 19:45:18.774136 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.773841    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fb9881b-1751-4e74-a32d-b0df066b0915-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb9881b-1751-4e74-a32d-b0df066b0915-maas-api-service-ca\") pod \"9fb9881b-1751-4e74-a32d-b0df066b0915\" (UID: \"9fb9881b-1751-4e74-a32d-b0df066b0915\") "
Oct 30 19:45:18.774179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.774134    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj5vp\" (UniqueName: \"kubernetes.io/projected/1be50a40-d610-4c64-90f6-408f40c81b79-kube-api-access-pj5vp\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.774179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.774155    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb9881b-1751-4e74-a32d-b0df066b0915-maas-api-service-ca" pod "9fb9881b-1751-4e74-a32d-b0df066b0915" (UID: "9fb9881b-1751-4e74-a32d-b0df066b0915"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:18.775752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.775729    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb9881b-1751-4e74-a32d-b0df066b0915-kube-api-access-gzrh6" pod "9fb9881b-1751-4e74-a32d-b0df066b0915" (UID: "9fb9881b-1751-4e74-a32d-b0df066b0915"). InnerVolumeSpecName "kube-api-access-gzrh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:18.875092 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.875049    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb9881b-1751-4e74-a32d-b0df066b0915-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.875092 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:18.875079    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzrh6\" (UniqueName: \"kubernetes.io/projected/9fb9881b-1751-4e74-a32d-b0df066b0915-kube-api-access-gzrh6\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:19.442952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.442920    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"
Oct 30 19:45:19.443145 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.442918    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt" event={"ID":"9fb9881b-1751-4e74-a32d-b0df066b0915","Type":"ContainerDied","Data":"810452cfaea3ff9151886b10fccbc75f5c8579d28b14dc8bb32b06fb497fba7d"}
Oct 30 19:45:19.443145 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.443083    5232 scope.go:122] "RemoveContainer" containerID="d03ca58e4e6d6ccc1302cdde8ca8db916b09e136d586f252b2faeb9066710360"
Oct 30 19:45:19.445669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.445639    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b" event={"ID":"1be50a40-d610-4c64-90f6-408f40c81b79","Type":"ContainerDied","Data":"a9654b16e4b4ed75d2a0c583865ad0752d55c0122e0c7de130835cf927c7d38f"}
Oct 30 19:45:19.445780 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.445654    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"
Oct 30 19:45:19.453548 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.453527    5232 scope.go:122] "RemoveContainer" containerID="d9ed956f6689b49c9352f7bb6a652c020f28a7281b22ea4f1e8e9c4c6177b9dd"
Oct 30 19:45:19.549670 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.546780    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" path="/var/lib/kubelet/pods/ed017d8a-96be-4e2e-951c-5a23391f1cac/volumes"
Oct 30 19:45:19.613964 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.613935    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"]
Oct 30 19:45:19.620047 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.620022    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889825-n95mt"]
Oct 30 19:45:19.751751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.751674    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"]
Oct 30 19:45:19.768395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:19.768364    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-59594f-29889825-gsk7b"]
Oct 30 19:45:21.540632 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:21.540552    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" path="/var/lib/kubelet/pods/1be50a40-d610-4c64-90f6-408f40c81b79/volumes"
Oct 30 19:45:21.541088 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:21.541042    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" path="/var/lib/kubelet/pods/9fb9881b-1751-4e74-a32d-b0df066b0915/volumes"
Oct 30 19:45:31.455581 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:31.455497    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"]
Oct 30 19:45:31.456054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:31.455720    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy" containerID="cri-o://301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1" gracePeriod=30
Oct 30 19:45:36.728391 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.728327    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:45:36.907887 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.907847    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istiod-ca-cert\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.908599 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.908575    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-token\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-token\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.909216 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.908477    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istiod-ca-cert" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:36.909421 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.909401    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-kube-api-access-2ld9s\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-kube-api-access-2ld9s\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.909595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.909571    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-podinfo\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.909725 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.909616    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-data\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.909725 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.909651    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-socket\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.909893 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.909730    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-credential-socket\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.909893 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.909783    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-certs\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.910033 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.910012    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-envoy\") pod \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\" (UID: \"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a\") "
Oct 30 19:45:36.910876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.910114    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-data" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:36.910876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.910383    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-credential-socket" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:36.910876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.910597    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:36.910876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.910620    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:36.910876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.910636    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:36.910876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.910844    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-socket" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:36.911309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.911104    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-certs" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:36.911507 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.911479    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-token" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:36.913020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.912994    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-podinfo" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:45:36.913190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.913142    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-envoy" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:36.913379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:36.913362    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-kube-api-access-2ld9s" pod "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" (UID: "1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a"). InnerVolumeSpecName "kube-api-access-2ld9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:37.012361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.012275    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:37.012361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.012303    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:37.012361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.012314    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ld9s\" (UniqueName: \"kubernetes.io/projected/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-kube-api-access-2ld9s\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:37.012361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.012323    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:37.012361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.012333    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:37.012361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.012342    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:37.570371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.570336    5232 generic.go:356] "Generic (PLEG): container finished" podID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerID="301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1" exitCode=0
Oct 30 19:45:37.570532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.570387    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"
Oct 30 19:45:37.570532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.570419    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" event={"ID":"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a","Type":"ContainerDied","Data":"301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1"}
Oct 30 19:45:37.570532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.570474    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g" event={"ID":"1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a","Type":"ContainerDied","Data":"dd91258235361920469ad257c43f177dd517905c709d33559ac112eef929d9a2"}
Oct 30 19:45:37.570532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.570496    5232 scope.go:122] "RemoveContainer" containerID="301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1"
Oct 30 19:45:37.584180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.584008    5232 scope.go:122] "RemoveContainer" containerID="301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1"
Oct 30 19:45:37.584378 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:37.584351    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1\": container with ID starting with 301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1 not found: ID does not exist" containerID="301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1"
Oct 30 19:45:37.584425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.584388    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1"} err="failed to get container status \"301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1\": rpc error: code = NotFound desc = could not find container \"301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1\": container with ID starting with 301f5162a75ee359a28d2fa0606ca5c1909ebfe46c388166c58fd49ea1e88aa1 not found: ID does not exist"
Oct 30 19:45:37.778226 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.778193    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"]
Oct 30 19:45:37.825452 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:37.825377    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-xqq7g"]
Oct 30 19:45:39.540178 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:45:39.540143    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" path="/var/lib/kubelet/pods/1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a/volumes"
Oct 30 19:45:46.187399 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:45:46.187362    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:46:16.188786 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:46:16.188743    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:46:19.863578 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.863544    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"]
Oct 30 19:46:19.864109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864095    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:46:19.864156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864111    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:46:19.864156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864122    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:46:19.864156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864127    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:46:19.864156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864142    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:46:19.864156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864147    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864159    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f6aa2ad7-53f3-43cc-bddf-9021e514b703" containerName="main"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864164    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6aa2ad7-53f3-43cc-bddf-9021e514b703" containerName="main"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864174    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864180    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864186    5232 cpu_manager.go:414] "containerMap: removing container" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864191    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864210    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864214    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864221    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864225    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864238    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864243    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864250    5232 cpu_manager.go:414] "containerMap: removing container" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864254    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864260    5232 cpu_manager.go:414] "containerMap: removing container" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864265    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864334    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:46:19.864337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864341    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864348    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1dfd836a-e7a7-4d9a-9cc3-a0e7f545bd7a" containerName="istio-proxy"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864356    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864361    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864366    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864372    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864377    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864383    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864390    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6aa2ad7-53f3-43cc-bddf-9021e514b703" containerName="main"
Oct 30 19:46:19.864924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.864396    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:46:19.867659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.867643    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:19.904539 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.904505    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:19.904695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.904547    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-metrics-certs\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:19.904695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.904652    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8658de33-feba-4517-b13b-c7deea58850c-openshift-service-ca\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:19.904695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.904677    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vb8z\" (UniqueName: \"kubernetes.io/projected/8658de33-feba-4517-b13b-c7deea58850c-kube-api-access-6vb8z\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:19.917948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.917918    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-f858d4-serving-cert\""
Oct 30 19:46:19.968777 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:19.968744    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"]
Oct 30 19:46:20.006537 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.006497    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.006741 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.006559    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-metrics-certs\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.006741 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.006638    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8658de33-feba-4517-b13b-c7deea58850c-openshift-service-ca\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.006741 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.006673    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vb8z\" (UniqueName: \"kubernetes.io/projected/8658de33-feba-4517-b13b-c7deea58850c-kube-api-access-6vb8z\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.006951 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:46:20.006902    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-serving-cert: secret "maas-api-e2e-autores-f858d4-serving-cert" not found
Oct 30 19:46:20.007006 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:46:20.006977    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls podName:8658de33-feba-4517-b13b-c7deea58850c nodeName:}" failed. No retries permitted until 2026-10-30 19:46:20.506953914 +0000 UTC m=+3573.091436908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls") pod "maas-api-e2e-autores-f858d4-678bf68978-hkztz" (UID: "8658de33-feba-4517-b13b-c7deea58850c") : secret "maas-api-e2e-autores-f858d4-serving-cert" not found
Oct 30 19:46:20.007857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.007790    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8658de33-feba-4517-b13b-c7deea58850c-openshift-service-ca\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.011300 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.011276    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-metrics-certs\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.068304 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.068273    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vb8z\" (UniqueName: \"kubernetes.io/projected/8658de33-feba-4517-b13b-c7deea58850c-kube-api-access-6vb8z\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.513764 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.513726    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.516137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.516115    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls\") pod \"maas-api-e2e-autores-f858d4-678bf68978-hkztz\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:20.780951 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:20.780864    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:21.159203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:21.159162    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"]
Oct 30 19:46:21.163339 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:46:21.163313    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8658de33_feba_4517_b13b_c7deea58850c.slice/crio-a2e48da9e11f418d27b1bd5dad330f49f1ba52d8abc9990015ea4d7785a245c8 WatchSource:0}: Error finding container a2e48da9e11f418d27b1bd5dad330f49f1ba52d8abc9990015ea4d7785a245c8: Status 404 returned error can't find the container with id a2e48da9e11f418d27b1bd5dad330f49f1ba52d8abc9990015ea4d7785a245c8
Oct 30 19:46:21.165168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:21.165150    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:46:21.901646 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:21.901604    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz" event={"ID":"8658de33-feba-4517-b13b-c7deea58850c","Type":"ContainerStarted","Data":"a2e48da9e11f418d27b1bd5dad330f49f1ba52d8abc9990015ea4d7785a245c8"}
Oct 30 19:46:22.909612 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:22.909565    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz" event={"ID":"8658de33-feba-4517-b13b-c7deea58850c","Type":"ContainerStarted","Data":"f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2"}
Oct 30 19:46:22.910115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:22.909670    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:23.025558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:23.025492    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz" podStartSLOduration=3.130693247 podStartE2EDuration="4.02547614s" totalImagesPullingTime="894.782893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:23.023965938 +0000 UTC m=+3575.608448935" watchObservedRunningTime="2026-10-30 19:46:23.02547614 +0000 UTC m=+3575.609959138"
Oct 30 19:46:28.921547 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:28.921517    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:46:46.187441 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:46:46.187401    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:46:48.104775 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:48.104740    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:46:48.109136 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:48.109115    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:46:48.124431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:48.124414    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:46:48.128381 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:46:48.128360    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:47:08.233642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:08.233606    5232 generic.go:356] "Generic (PLEG): container finished" podID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerID="e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4" exitCode=0
Oct 30 19:47:08.234152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:08.233660    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" event={"ID":"d21b1b6c-bcf5-4787-8c1b-c055df3b6264","Type":"ContainerDied","Data":"e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4"}
Oct 30 19:47:08.234152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:08.234086    5232 scope.go:122] "RemoveContainer" containerID="e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4"
Oct 30 19:47:08.235910 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:08.235877    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:08.236080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:08.235971    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:08.236080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:08.236021    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:09.269541 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:09.269504    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" event={"ID":"d21b1b6c-bcf5-4787-8c1b-c055df3b6264","Type":"ContainerStarted","Data":"67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e"}
Oct 30 19:47:09.757168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:09.757133    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:47:09.757563 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:09.757540    5232 patch_prober.go:29] interesting pod/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:47:09.757752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:09.757725    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:47:10.761978 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:10.761936    5232 patch_prober.go:29] interesting pod/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:47:10.762491 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:10.762000    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:47:11.748184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:11.748148    5232 patch_prober.go:29] interesting pod/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:47:11.748506 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:11.748466    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:47:12.760952 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:12.760913    5232 patch_prober.go:29] interesting pod/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:47:12.762000 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:12.761947    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:47:13.762309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:13.762222    5232 patch_prober.go:29] interesting pod/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:47:13.762309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:13.762298    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:47:14.748540 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:14.748503    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:47:14.749025 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:14.749004    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:47:14.749648 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:47:14.749629    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:47:16.187796 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:47:16.187760    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:47:46.193961 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:47:46.193916    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:48:16.187499 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:48:16.187454    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:48:40.374348 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.374243    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"]
Oct 30 19:48:40.375007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.374493    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b" podUID="51302f02-327c-45bc-9d51-5e11a5892b22" containerName="maas-api" containerID="cri-o://5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1" gracePeriod=30
Oct 30 19:48:40.647105 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.647078    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:48:40.813745 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.813715    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls\") pod \"51302f02-327c-45bc-9d51-5e11a5892b22\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") "
Oct 30 19:48:40.813977 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.813774    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51302f02-327c-45bc-9d51-5e11a5892b22-kube-api-access-9jcsn\" (UniqueName: \"kubernetes.io/projected/51302f02-327c-45bc-9d51-5e11a5892b22-kube-api-access-9jcsn\") pod \"51302f02-327c-45bc-9d51-5e11a5892b22\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") "
Oct 30 19:48:40.813977 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.813802    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-metrics-certs\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-metrics-certs\") pod \"51302f02-327c-45bc-9d51-5e11a5892b22\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") "
Oct 30 19:48:40.813977 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.813854    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51302f02-327c-45bc-9d51-5e11a5892b22-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51302f02-327c-45bc-9d51-5e11a5892b22-openshift-service-ca\") pod \"51302f02-327c-45bc-9d51-5e11a5892b22\" (UID: \"51302f02-327c-45bc-9d51-5e11a5892b22\") "
Oct 30 19:48:40.814295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.814269    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51302f02-327c-45bc-9d51-5e11a5892b22-openshift-service-ca" pod "51302f02-327c-45bc-9d51-5e11a5892b22" (UID: "51302f02-327c-45bc-9d51-5e11a5892b22"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:40.814518 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.814496    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51302f02-327c-45bc-9d51-5e11a5892b22-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:40.816372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.816330    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls" pod "51302f02-327c-45bc-9d51-5e11a5892b22" (UID: "51302f02-327c-45bc-9d51-5e11a5892b22"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:40.816494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.816414    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-metrics-certs" pod "51302f02-327c-45bc-9d51-5e11a5892b22" (UID: "51302f02-327c-45bc-9d51-5e11a5892b22"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:40.816494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.816439    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51302f02-327c-45bc-9d51-5e11a5892b22-kube-api-access-9jcsn" pod "51302f02-327c-45bc-9d51-5e11a5892b22" (UID: "51302f02-327c-45bc-9d51-5e11a5892b22"). InnerVolumeSpecName "kube-api-access-9jcsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:40.889655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.889457    5232 generic.go:356] "Generic (PLEG): container finished" podID="51302f02-327c-45bc-9d51-5e11a5892b22" containerID="5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1" exitCode=0
Oct 30 19:48:40.889655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.889520    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"
Oct 30 19:48:40.889655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.889545    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b" event={"ID":"51302f02-327c-45bc-9d51-5e11a5892b22","Type":"ContainerDied","Data":"5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1"}
Oct 30 19:48:40.889655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.889612    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b" event={"ID":"51302f02-327c-45bc-9d51-5e11a5892b22","Type":"ContainerDied","Data":"791b4fa355b44fa7e99211b8afd0651f030c820d10f9d37e34ed87f09aef1cc5"}
Oct 30 19:48:40.890054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.889657    5232 scope.go:122] "RemoveContainer" containerID="5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1"
Oct 30 19:48:40.901567 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.901547    5232 scope.go:122] "RemoveContainer" containerID="5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1"
Oct 30 19:48:40.901912 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:48:40.901887    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1\": container with ID starting with 5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1 not found: ID does not exist" containerID="5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1"
Oct 30 19:48:40.901981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.901922    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1"} err="failed to get container status \"5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1\": rpc error: code = NotFound desc = could not find container \"5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1\": container with ID starting with 5dd95ef7315dab89f3667d18e077609cee389259ddfea66e06a356ee679816e1 not found: ID does not exist"
Oct 30 19:48:40.915464 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.915436    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:40.915585 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.915467    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jcsn\" (UniqueName: \"kubernetes.io/projected/51302f02-327c-45bc-9d51-5e11a5892b22-kube-api-access-9jcsn\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:40.915585 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:40.915483    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51302f02-327c-45bc-9d51-5e11a5892b22-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:41.049316 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:41.049275    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"]
Oct 30 19:48:41.054284 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:41.054257    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-af72cf-f54654c76-9l94b"]
Oct 30 19:48:41.541314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:41.541281    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51302f02-327c-45bc-9d51-5e11a5892b22" path="/var/lib/kubelet/pods/51302f02-327c-45bc-9d51-5e11a5892b22/volumes"
Oct 30 19:48:46.187558 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:48:46.187527    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:48:57.840665 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:48:57.840629    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"]
Oct 30 19:49:12.860882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:12.860837    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd" podUID="b72541bf-5382-487a-a562-3015b45f188a" containerName="main" containerID="cri-o://e83df5ef6a0bbcc5b267486c8690c59658bd8664e21c0f6f8928a18db65d9d2d" gracePeriod=45
Oct 30 19:49:13.111278 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.111199    5232 generic.go:356] "Generic (PLEG): container finished" podID="b72541bf-5382-487a-a562-3015b45f188a" containerID="e83df5ef6a0bbcc5b267486c8690c59658bd8664e21c0f6f8928a18db65d9d2d" exitCode=0
Oct 30 19:49:13.111437 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.111282    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd" event={"ID":"b72541bf-5382-487a-a562-3015b45f188a","Type":"ContainerDied","Data":"e83df5ef6a0bbcc5b267486c8690c59658bd8664e21c0f6f8928a18db65d9d2d"}
Oct 30 19:49:13.141569 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.141545    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:49:13.216677 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.216646    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-model-cache\") pod \"b72541bf-5382-487a-a562-3015b45f188a\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") "
Oct 30 19:49:13.216920 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.216802    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-home\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-home\") pod \"b72541bf-5382-487a-a562-3015b45f188a\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") "
Oct 30 19:49:13.216920 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.216904    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b72541bf-5382-487a-a562-3015b45f188a-kube-api-access-8t2f2\" (UniqueName: \"kubernetes.io/projected/b72541bf-5382-487a-a562-3015b45f188a-kube-api-access-8t2f2\") pod \"b72541bf-5382-487a-a562-3015b45f188a\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") "
Oct 30 19:49:13.217064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.216945    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-tmp-dir\") pod \"b72541bf-5382-487a-a562-3015b45f188a\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") "
Oct 30 19:49:13.217064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.216978    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-dshm\") pod \"b72541bf-5382-487a-a562-3015b45f188a\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") "
Oct 30 19:49:13.217064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.216988    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-model-cache" pod "b72541bf-5382-487a-a562-3015b45f188a" (UID: "b72541bf-5382-487a-a562-3015b45f188a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:13.217064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.217004    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b72541bf-5382-487a-a562-3015b45f188a-tls-certs\" (UniqueName: \"kubernetes.io/secret/b72541bf-5382-487a-a562-3015b45f188a-tls-certs\") pod \"b72541bf-5382-487a-a562-3015b45f188a\" (UID: \"b72541bf-5382-487a-a562-3015b45f188a\") "
Oct 30 19:49:13.217064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.217020    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-home" pod "b72541bf-5382-487a-a562-3015b45f188a" (UID: "b72541bf-5382-487a-a562-3015b45f188a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:13.217341 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.217227    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-tmp-dir" pod "b72541bf-5382-487a-a562-3015b45f188a" (UID: "b72541bf-5382-487a-a562-3015b45f188a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:13.217471 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.217452    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.217536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.217476    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.217536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.217489    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.219115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.219094    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-dshm" pod "b72541bf-5382-487a-a562-3015b45f188a" (UID: "b72541bf-5382-487a-a562-3015b45f188a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:13.219240 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.219218    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72541bf-5382-487a-a562-3015b45f188a-tls-certs" pod "b72541bf-5382-487a-a562-3015b45f188a" (UID: "b72541bf-5382-487a-a562-3015b45f188a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:13.219298 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.219246    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72541bf-5382-487a-a562-3015b45f188a-kube-api-access-8t2f2" pod "b72541bf-5382-487a-a562-3015b45f188a" (UID: "b72541bf-5382-487a-a562-3015b45f188a"). InnerVolumeSpecName "kube-api-access-8t2f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:13.318437 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.318398    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t2f2\" (UniqueName: \"kubernetes.io/projected/b72541bf-5382-487a-a562-3015b45f188a-kube-api-access-8t2f2\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.318437 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.318434    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b72541bf-5382-487a-a562-3015b45f188a-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.318437 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:13.318444    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b72541bf-5382-487a-a562-3015b45f188a-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:14.118767 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.118737    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"
Oct 30 19:49:14.119214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.118756    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd" event={"ID":"b72541bf-5382-487a-a562-3015b45f188a","Type":"ContainerDied","Data":"668fb2c2dc563bfab4d7ce678825135802cbad3d150f92f006924181ccf0008c"}
Oct 30 19:49:14.119214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.118835    5232 scope.go:122] "RemoveContainer" containerID="e83df5ef6a0bbcc5b267486c8690c59658bd8664e21c0f6f8928a18db65d9d2d"
Oct 30 19:49:14.278474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.278439    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"]
Oct 30 19:49:14.325696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.325663    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-85f471/auth-test-model-85f471-kserve-6d66996db5-wqndd"]
Oct 30 19:49:14.647197 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.647160    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"]
Oct 30 19:49:14.647960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.647934    5232 cpu_manager.go:414] "containerMap: removing container" podUID="b72541bf-5382-487a-a562-3015b45f188a" containerName="main"
Oct 30 19:49:14.647960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.647959    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b72541bf-5382-487a-a562-3015b45f188a" containerName="main"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.647992    5232 cpu_manager.go:414] "containerMap: removing container" podUID="51302f02-327c-45bc-9d51-5e11a5892b22" containerName="maas-api"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.647999    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51302f02-327c-45bc-9d51-5e11a5892b22" containerName="maas-api"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648010    5232 cpu_manager.go:414] "containerMap: removing container" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648018    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fb9881b-1751-4e74-a32d-b0df066b0915" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648057    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648064    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648103    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648110    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648274    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fdb9572-81ee-49b7-aded-db90d1cf5d02" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648295    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1be50a40-d610-4c64-90f6-408f40c81b79" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648306    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed017d8a-96be-4e2e-951c-5a23391f1cac" containerName="cleanup"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648314    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="b72541bf-5382-487a-a562-3015b45f188a" containerName="main"
Oct 30 19:49:14.648355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.648330    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="51302f02-327c-45bc-9d51-5e11a5892b22" containerName="maas-api"
Oct 30 19:49:14.653137 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.653117    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.703523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.703490    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-17216f-serving-cert\""
Oct 30 19:49:14.712290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.712263    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"]
Oct 30 19:49:14.735867 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.735807    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.736052 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.735929    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.736052 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.735997    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2dq4\" (UniqueName: \"kubernetes.io/projected/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-kube-api-access-s2dq4\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.736161 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.736099    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-metrics-certs\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.837919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.837880    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-metrics-certs\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.838146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.838028    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.838146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.838062    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.838146 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.838079    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2dq4\" (UniqueName: \"kubernetes.io/projected/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-kube-api-access-s2dq4\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.838332 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:49:14.838187    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-serving-cert: secret "maas-api-e2e-extmod-a-17216f-serving-cert" not found
Oct 30 19:49:14.838332 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:49:14.838279    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls podName:5ba1a1aa-85f3-4750-ba11-a3ac1b686779 nodeName:}" failed. No retries permitted until 2026-10-30 19:49:15.338257092 +0000 UTC m=+3747.922740072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls") pod "maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf" (UID: "5ba1a1aa-85f3-4750-ba11-a3ac1b686779") : secret "maas-api-e2e-extmod-a-17216f-serving-cert" not found
Oct 30 19:49:14.838741 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.838711    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.840474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.840455    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-metrics-certs\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:14.900805 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:14.900726    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2dq4\" (UniqueName: \"kubernetes.io/projected/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-kube-api-access-s2dq4\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:15.345163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:15.345128    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:15.347751 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:15.347728    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls\") pod \"maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:15.541410 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:15.541372    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b72541bf-5382-487a-a562-3015b45f188a" path="/var/lib/kubelet/pods/b72541bf-5382-487a-a562-3015b45f188a/volumes"
Oct 30 19:49:15.567263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:15.567233    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:15.947123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:15.947013    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"]
Oct 30 19:49:15.952407 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:49:15.952370    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba1a1aa_85f3_4750_ba11_a3ac1b686779.slice/crio-fae7f11eb12fdffd6611fb01415ea2ce77dbf8d45a9a121cdf6a0f0e25e250ad WatchSource:0}: Error finding container fae7f11eb12fdffd6611fb01415ea2ce77dbf8d45a9a121cdf6a0f0e25e250ad: Status 404 returned error can't find the container with id fae7f11eb12fdffd6611fb01415ea2ce77dbf8d45a9a121cdf6a0f0e25e250ad
Oct 30 19:49:16.133769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:16.133731    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf" event={"ID":"5ba1a1aa-85f3-4750-ba11-a3ac1b686779","Type":"ContainerStarted","Data":"fae7f11eb12fdffd6611fb01415ea2ce77dbf8d45a9a121cdf6a0f0e25e250ad"}
Oct 30 19:49:16.186972 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:49:16.186934    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:49:17.144204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:17.144164    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf" event={"ID":"5ba1a1aa-85f3-4750-ba11-a3ac1b686779","Type":"ContainerStarted","Data":"f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2"}
Oct 30 19:49:17.144607 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:17.144229    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:17.259934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:17.259882    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf" podStartSLOduration=2.430765916 podStartE2EDuration="3.259865363s" totalImagesPullingTime="829.099447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:17.257603745 +0000 UTC m=+3749.842086743" watchObservedRunningTime="2026-10-30 19:49:17.259865363 +0000 UTC m=+3749.844348359"
Oct 30 19:49:23.157388 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:49:23.157357    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:49:46.188231 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:49:46.188186    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:50:00.880538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:00.880460    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"]
Oct 30 19:50:00.881053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:00.880675    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy" containerID="cri-o://0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709" gracePeriod=30
Oct 30 19:50:06.152016 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.151943    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:50:06.285688 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.285650    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-credential-socket\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.285911 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.285714    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-certs\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.285911 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.285788    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-kube-api-access-cvtj4\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-kube-api-access-cvtj4\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.285911 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.285852    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-data\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.286095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.285913    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-socket\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.286095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.285940    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-istio-token\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-istio-token\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.286095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.285979    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81846c84-b541-44b0-84e3-f0063c18d613-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81846c84-b541-44b0-84e3-f0063c18d613-istio-podinfo\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.286095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286008    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81846c84-b541-44b0-84e3-f0063c18d613-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81846c84-b541-44b0-84e3-f0063c18d613-istiod-ca-cert\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.286095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286034    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-envoy\") pod \"81846c84-b541-44b0-84e3-f0063c18d613\" (UID: \"81846c84-b541-44b0-84e3-f0063c18d613\") "
Oct 30 19:50:06.286095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286036    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-credential-socket" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:06.286408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286105    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-certs" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:06.286408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286230    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-data" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:06.286408 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286252    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-socket" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:06.286537 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286446    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81846c84-b541-44b0-84e3-f0063c18d613-istiod-ca-cert" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:50:06.286662 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286640    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.286768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286669    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.286768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286684    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.286768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286699    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.286768 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.286712    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81846c84-b541-44b0-84e3-f0063c18d613-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.288555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.288522    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-envoy" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:06.288555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.288524    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-istio-token" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:06.288742 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.288585    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81846c84-b541-44b0-84e3-f0063c18d613-istio-podinfo" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:50:06.288851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.288796    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-kube-api-access-cvtj4" pod "81846c84-b541-44b0-84e3-f0063c18d613" (UID: "81846c84-b541-44b0-84e3-f0063c18d613"). InnerVolumeSpecName "kube-api-access-cvtj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:06.388514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.388476    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvtj4\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-kube-api-access-cvtj4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.388514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.388510    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81846c84-b541-44b0-84e3-f0063c18d613-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.388514 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.388520    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81846c84-b541-44b0-84e3-f0063c18d613-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.388754 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.388531    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81846c84-b541-44b0-84e3-f0063c18d613-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:06.475980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.475880    5232 generic.go:356] "Generic (PLEG): container finished" podID="81846c84-b541-44b0-84e3-f0063c18d613" containerID="0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709" exitCode=0
Oct 30 19:50:06.475980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.475952    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"
Oct 30 19:50:06.475980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.475959    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" event={"ID":"81846c84-b541-44b0-84e3-f0063c18d613","Type":"ContainerDied","Data":"0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709"}
Oct 30 19:50:06.476262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.476015    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5" event={"ID":"81846c84-b541-44b0-84e3-f0063c18d613","Type":"ContainerDied","Data":"d901888c2c932d019ec7e46ef4d9b771b8015246ae1e67c205f215b360923843"}
Oct 30 19:50:06.476262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.476038    5232 scope.go:122] "RemoveContainer" containerID="0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709"
Oct 30 19:50:06.487218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.487188    5232 scope.go:122] "RemoveContainer" containerID="ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22"
Oct 30 19:50:06.498188 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.498161    5232 scope.go:122] "RemoveContainer" containerID="0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709"
Oct 30 19:50:06.498534 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:06.498513    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709\": container with ID starting with 0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709 not found: ID does not exist" containerID="0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709"
Oct 30 19:50:06.498621 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.498545    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709"} err="failed to get container status \"0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709\": rpc error: code = NotFound desc = could not find container \"0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709\": container with ID starting with 0db3bb2ea71a54c832b43b2efd6a910035b53ea9f4418099233500b417a14709 not found: ID does not exist"
Oct 30 19:50:06.498621 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.498566    5232 scope.go:122] "RemoveContainer" containerID="ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22"
Oct 30 19:50:06.498866 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:06.498846    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22\": container with ID starting with ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22 not found: ID does not exist" containerID="ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22"
Oct 30 19:50:06.498912 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.498874    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22"} err="failed to get container status \"ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22\": rpc error: code = NotFound desc = could not find container \"ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22\": container with ID starting with ae683b63c9af86f20f4fc175a6a861eb77430c284279afe6125e0c88c101ba22 not found: ID does not exist"
Oct 30 19:50:06.637020 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.636980    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"]
Oct 30 19:50:06.642204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:06.642172    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-af72cf-openshift-default-57bf445849-5qdm5"]
Oct 30 19:50:07.541237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:07.541200    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81846c84-b541-44b0-84e3-f0063c18d613" path="/var/lib/kubelet/pods/81846c84-b541-44b0-84e3-f0063c18d613/volumes"
Oct 30 19:50:11.031760 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.031717    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"]
Oct 30 19:50:11.032305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.031995    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz" podUID="8658de33-feba-4517-b13b-c7deea58850c" containerName="maas-api" containerID="cri-o://f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2" gracePeriod=30
Oct 30 19:50:11.299303 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.299235    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:50:11.464083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.464049    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls\") pod \"8658de33-feba-4517-b13b-c7deea58850c\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") "
Oct 30 19:50:11.464295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.464115    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8658de33-feba-4517-b13b-c7deea58850c-kube-api-access-6vb8z\" (UniqueName: \"kubernetes.io/projected/8658de33-feba-4517-b13b-c7deea58850c-kube-api-access-6vb8z\") pod \"8658de33-feba-4517-b13b-c7deea58850c\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") "
Oct 30 19:50:11.464295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.464157    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8658de33-feba-4517-b13b-c7deea58850c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8658de33-feba-4517-b13b-c7deea58850c-openshift-service-ca\") pod \"8658de33-feba-4517-b13b-c7deea58850c\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") "
Oct 30 19:50:11.464295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.464215    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-metrics-certs\") pod \"8658de33-feba-4517-b13b-c7deea58850c\" (UID: \"8658de33-feba-4517-b13b-c7deea58850c\") "
Oct 30 19:50:11.464613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.464583    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8658de33-feba-4517-b13b-c7deea58850c-openshift-service-ca" pod "8658de33-feba-4517-b13b-c7deea58850c" (UID: "8658de33-feba-4517-b13b-c7deea58850c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:50:11.466243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.466216    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-metrics-certs" pod "8658de33-feba-4517-b13b-c7deea58850c" (UID: "8658de33-feba-4517-b13b-c7deea58850c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:11.466330 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.466255    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls" pod "8658de33-feba-4517-b13b-c7deea58850c" (UID: "8658de33-feba-4517-b13b-c7deea58850c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:11.466462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.466325    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8658de33-feba-4517-b13b-c7deea58850c-kube-api-access-6vb8z" pod "8658de33-feba-4517-b13b-c7deea58850c" (UID: "8658de33-feba-4517-b13b-c7deea58850c"). InnerVolumeSpecName "kube-api-access-6vb8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:11.510534 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.510497    5232 generic.go:356] "Generic (PLEG): container finished" podID="8658de33-feba-4517-b13b-c7deea58850c" containerID="f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2" exitCode=0
Oct 30 19:50:11.510707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.510549    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"
Oct 30 19:50:11.510707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.510585    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz" event={"ID":"8658de33-feba-4517-b13b-c7deea58850c","Type":"ContainerDied","Data":"f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2"}
Oct 30 19:50:11.510707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.510644    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz" event={"ID":"8658de33-feba-4517-b13b-c7deea58850c","Type":"ContainerDied","Data":"a2e48da9e11f418d27b1bd5dad330f49f1ba52d8abc9990015ea4d7785a245c8"}
Oct 30 19:50:11.510707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.510671    5232 scope.go:122] "RemoveContainer" containerID="f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2"
Oct 30 19:50:11.521972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.521948    5232 scope.go:122] "RemoveContainer" containerID="f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2"
Oct 30 19:50:11.522283 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:11.522257    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2\": container with ID starting with f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2 not found: ID does not exist" containerID="f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2"
Oct 30 19:50:11.522369 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.522295    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2"} err="failed to get container status \"f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2\": rpc error: code = NotFound desc = could not find container \"f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2\": container with ID starting with f2e403bfbda01fe1f81ab4174c4e197cd3fdd9510deb47c36978fb95e0830ef2 not found: ID does not exist"
Oct 30 19:50:11.565987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.565887    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vb8z\" (UniqueName: \"kubernetes.io/projected/8658de33-feba-4517-b13b-c7deea58850c-kube-api-access-6vb8z\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:11.565987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.565917    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8658de33-feba-4517-b13b-c7deea58850c-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:11.565987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.565927    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:11.565987 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.565937    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8658de33-feba-4517-b13b-c7deea58850c-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:11.670358 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.670324    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"]
Oct 30 19:50:11.675584 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:11.675545    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f858d4-678bf68978-hkztz"]
Oct 30 19:50:13.543077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:13.542996    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8658de33-feba-4517-b13b-c7deea58850c" path="/var/lib/kubelet/pods/8658de33-feba-4517-b13b-c7deea58850c/volumes"
Oct 30 19:50:16.191859 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:16.191779    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:50:20.880179 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:20.880145    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"]
Oct 30 19:50:20.880601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:20.880355    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy" containerID="cri-o://67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e" gracePeriod=30
Oct 30 19:50:26.146562 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.146487    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:50:26.269441 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269404    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-kube-api-access-4pvrr\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-kube-api-access-4pvrr\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269481    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-socket\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269585    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-data\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269628    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-envoy\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269717    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istiod-ca-cert\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269751    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-certs\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269798    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-credential-socket\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269853    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-socket" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:26.269995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269899    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-token\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-token\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269941    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-podinfo\") pod \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\" (UID: \"d21b1b6c-bcf5-4787-8c1b-c055df3b6264\") "
Oct 30 19:50:26.269995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.269894    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-data" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:26.270218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.270196    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-credential-socket" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:26.270276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.270229    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istiod-ca-cert" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:50:26.270410 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.270382    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-certs" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:26.270587 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.270571    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.270669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.270590    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.270669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.270599    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.270669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.270607    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.270669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.270616    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.272030 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.272010    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-token" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:26.272139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.272038    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-kube-api-access-4pvrr" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "kube-api-access-4pvrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:26.272211 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.272193    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-envoy" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:26.272304 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.272281    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-podinfo" pod "d21b1b6c-bcf5-4787-8c1b-c055df3b6264" (UID: "d21b1b6c-bcf5-4787-8c1b-c055df3b6264"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:50:26.372098 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.372063    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.372098 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.372092    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.372098 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.372103    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pvrr\" (UniqueName: \"kubernetes.io/projected/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-kube-api-access-4pvrr\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.372349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.372116    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21b1b6c-bcf5-4787-8c1b-c055df3b6264-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.617218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.617184    5232 generic.go:356] "Generic (PLEG): container finished" podID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerID="67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e" exitCode=0
Oct 30 19:50:26.617423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.617252    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"
Oct 30 19:50:26.617423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.617275    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" event={"ID":"d21b1b6c-bcf5-4787-8c1b-c055df3b6264","Type":"ContainerDied","Data":"67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e"}
Oct 30 19:50:26.617423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.617333    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m" event={"ID":"d21b1b6c-bcf5-4787-8c1b-c055df3b6264","Type":"ContainerDied","Data":"2111e37a554741d77ad5dcee7e624aaeab87251d1356983774e256f83b782441"}
Oct 30 19:50:26.617423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.617363    5232 scope.go:122] "RemoveContainer" containerID="67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e"
Oct 30 19:50:26.632393 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.632369    5232 scope.go:122] "RemoveContainer" containerID="e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4"
Oct 30 19:50:26.641939 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.641919    5232 scope.go:122] "RemoveContainer" containerID="67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e"
Oct 30 19:50:26.642245 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:26.642219    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e\": container with ID starting with 67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e not found: ID does not exist" containerID="67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e"
Oct 30 19:50:26.642339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.642251    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e"} err="failed to get container status \"67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e\": rpc error: code = NotFound desc = could not find container \"67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e\": container with ID starting with 67c16c1e6fe09709684dc7a87e8d17f49d9d9313442e4ab9622bd87f09c92b3e not found: ID does not exist"
Oct 30 19:50:26.642339 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.642272    5232 scope.go:122] "RemoveContainer" containerID="e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4"
Oct 30 19:50:26.642572 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:26.642553    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4\": container with ID starting with e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4 not found: ID does not exist" containerID="e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4"
Oct 30 19:50:26.642634 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.642587    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4"} err="failed to get container status \"e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4\": rpc error: code = NotFound desc = could not find container \"e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4\": container with ID starting with e8975b9144088f62497c71002c7e8797bbb90ab461ea8dc4738c6e31f11026d4 not found: ID does not exist"
Oct 30 19:50:26.777729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.777688    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"]
Oct 30 19:50:26.785710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:26.785680    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-f858d4-openshift-default-8b78cc457-86g9m"]
Oct 30 19:50:27.547022 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:27.546991    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" path="/var/lib/kubelet/pods/d21b1b6c-bcf5-4787-8c1b-c055df3b6264/volumes"
Oct 30 19:50:28.960894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.960856    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"]
Oct 30 19:50:28.961508 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961491    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy"
Oct 30 19:50:28.961604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961512    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy"
Oct 30 19:50:28.961604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961532    5232 cpu_manager.go:414] "containerMap: removing container" podUID="8658de33-feba-4517-b13b-c7deea58850c" containerName="maas-api"
Oct 30 19:50:28.961604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961538    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8658de33-feba-4517-b13b-c7deea58850c" containerName="maas-api"
Oct 30 19:50:28.961604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961553    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy"
Oct 30 19:50:28.961604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961560    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy"
Oct 30 19:50:28.961604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961587    5232 cpu_manager.go:414] "containerMap: removing container" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy"
Oct 30 19:50:28.961604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961594    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy"
Oct 30 19:50:28.961604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961605    5232 cpu_manager.go:414] "containerMap: removing container" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy"
Oct 30 19:50:28.962019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961612    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy"
Oct 30 19:50:28.962019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961737    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy"
Oct 30 19:50:28.962019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961750    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="81846c84-b541-44b0-84e3-f0063c18d613" containerName="istio-proxy"
Oct 30 19:50:28.962019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961765    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy"
Oct 30 19:50:28.962019 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.961775    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="8658de33-feba-4517-b13b-c7deea58850c" containerName="maas-api"
Oct 30 19:50:28.967113 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:28.967090    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.069237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.069199    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"]
Oct 30 19:50:29.089294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.089260    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-b330be-openshift-default-dockercfg-w965c\""
Oct 30 19:50:29.137214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137179    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-envoy\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.137214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137216    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/098d5771-830f-45a6-acda-76ca9d7dd365-istio-podinfo\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.137434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137316    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-data\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.137434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137353    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.137434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137412    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/098d5771-830f-45a6-acda-76ca9d7dd365-istiod-ca-cert\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.137538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137433    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pths\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-kube-api-access-9pths\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.137538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137527    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-istio-token\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.137606 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137577    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-credential-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.137641 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.137630    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-certs\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238524    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-certs\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238591    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-envoy\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238614    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/098d5771-830f-45a6-acda-76ca9d7dd365-istio-podinfo\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238663    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-data\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238688    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238726    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/098d5771-830f-45a6-acda-76ca9d7dd365-istiod-ca-cert\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238756    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pths\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-kube-api-access-9pths\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238866    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-istio-token\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.238942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238919    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-credential-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.239306 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.238993    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-certs\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.239306 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.239065    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.239432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.239306    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-credential-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.239432 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.239354    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-data\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.239561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.239534    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/098d5771-830f-45a6-acda-76ca9d7dd365-istiod-ca-cert\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.241007 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.240984    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-envoy\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.241253 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.241235    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/098d5771-830f-45a6-acda-76ca9d7dd365-istio-podinfo\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.297953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.297921    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-istio-token\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.298119 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.298065    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pths\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-kube-api-access-9pths\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:29.580913 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:29.580804    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:30.008653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:30.008615    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"]
Oct 30 19:50:30.020605 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:50:30.020577    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098d5771_830f_45a6_acda_76ca9d7dd365.slice/crio-84bec3062b545433054947d5175618c7a3b04c4e50f455ae19927461ebb07044 WatchSource:0}: Error finding container 84bec3062b545433054947d5175618c7a3b04c4e50f455ae19927461ebb07044: Status 404 returned error can't find the container with id 84bec3062b545433054947d5175618c7a3b04c4e50f455ae19927461ebb07044
Oct 30 19:50:30.023191 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:30.023153    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:30.023275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:30.023229    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:30.023275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:30.023262    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:30.660650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:30.660618    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" event={"ID":"098d5771-830f-45a6-acda-76ca9d7dd365","Type":"ContainerStarted","Data":"c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2"}
Oct 30 19:50:30.660857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:30.660674    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" event={"ID":"098d5771-830f-45a6-acda-76ca9d7dd365","Type":"ContainerStarted","Data":"84bec3062b545433054947d5175618c7a3b04c4e50f455ae19927461ebb07044"}
Oct 30 19:50:30.873424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:30.873367    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podStartSLOduration=2.873351361 podStartE2EDuration="2.873351361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:30.825547378 +0000 UTC m=+3823.410030377" watchObservedRunningTime="2026-10-30 19:50:30.873351361 +0000 UTC m=+3823.457834358"
Oct 30 19:50:31.581753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:31.581721    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:31.587748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:31.587724    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:31.668131 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:31.668095    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:31.676654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:31.676621    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:50:38.560996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.560966    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"]
Oct 30 19:50:38.562856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.562830    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d21b1b6c-bcf5-4787-8c1b-c055df3b6264" containerName="istio-proxy"
Oct 30 19:50:38.566469 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.566451    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.616638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.616605    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-4a36b7-openshift-default-dockercfg-c9cs2\""
Oct 30 19:50:38.647853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.647761    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-data\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.648133 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.648105    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-certs\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.648288 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.648152    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-podinfo\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.648364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.648312    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn5mb\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-kube-api-access-tn5mb\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.648364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.648353    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/494821ca-b40d-4423-b192-0ce5a101eeb4-istiod-ca-cert\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.648497 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.648449    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-credential-socket\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.648497 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.648477    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-envoy\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.648612 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.648563    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-socket\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.648698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.648682    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-token\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.672716 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.672691    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"]
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750257    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-token\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750302    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-data\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750327    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-certs\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750347    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-podinfo\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750387    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn5mb\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-kube-api-access-tn5mb\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750406    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/494821ca-b40d-4423-b192-0ce5a101eeb4-istiod-ca-cert\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750435    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-credential-socket\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750468    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-envoy\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.751144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.750504    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-socket\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.752968 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.752931    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-certs\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.753664 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.753643    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/494821ca-b40d-4423-b192-0ce5a101eeb4-istiod-ca-cert\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.754342 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.754303    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-credential-socket\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.755122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.755098    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-socket\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.755377 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.755350    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-data\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.756715 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.756672    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-envoy\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.760243 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.760218    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-podinfo\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.815621 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.815547    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-token\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.815621 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.815588    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn5mb\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-kube-api-access-tn5mb\") pod \"e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") " pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:38.880326 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:38.880290    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:39.327208 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:39.327170    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"]
Oct 30 19:50:39.331522 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:50:39.331491    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494821ca_b40d_4423_b192_0ce5a101eeb4.slice/crio-1b8be4a488c0a6aca6c17df0ffd33e72de9d5ff0340c760b40cdeef9724b642b WatchSource:0}: Error finding container 1b8be4a488c0a6aca6c17df0ffd33e72de9d5ff0340c760b40cdeef9724b642b: Status 404 returned error can't find the container with id 1b8be4a488c0a6aca6c17df0ffd33e72de9d5ff0340c760b40cdeef9724b642b
Oct 30 19:50:39.334044 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:39.334011    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:39.334116 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:39.334083    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:39.334116 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:39.334112    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:39.725883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:39.725840    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" event={"ID":"494821ca-b40d-4423-b192-0ce5a101eeb4","Type":"ContainerStarted","Data":"c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6"}
Oct 30 19:50:39.726349 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:39.725908    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" event={"ID":"494821ca-b40d-4423-b192-0ce5a101eeb4","Type":"ContainerStarted","Data":"1b8be4a488c0a6aca6c17df0ffd33e72de9d5ff0340c760b40cdeef9724b642b"}
Oct 30 19:50:39.846582 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:39.846525    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" podStartSLOduration=1.846512072 podStartE2EDuration="1.846512072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:39.8436297 +0000 UTC m=+3832.428112700" watchObservedRunningTime="2026-10-30 19:50:39.846512072 +0000 UTC m=+3832.430995069"
Oct 30 19:50:39.881062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:39.881028    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:40.886532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:40.886493    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:41.740110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:41.740070    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:41.741083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:41.741064    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:50:46.188242 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:46.188198    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:50:47.606440 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.606404    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"]
Oct 30 19:50:47.611257 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.611235    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.661377 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.661345    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-660b59-serving-cert\""
Oct 30 19:50:47.661884 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.661858    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-metrics-certs\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.661988 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.661968    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq8j9\" (UniqueName: \"kubernetes.io/projected/519960a2-3a77-4228-8f22-0455fcd3cfa9-kube-api-access-kq8j9\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.662062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.662050    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.662097 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.662088    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/519960a2-3a77-4228-8f22-0455fcd3cfa9-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.712919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.712879    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"]
Oct 30 19:50:47.763685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.763642    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-metrics-certs\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.763896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.763728    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq8j9\" (UniqueName: \"kubernetes.io/projected/519960a2-3a77-4228-8f22-0455fcd3cfa9-kube-api-access-kq8j9\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.763896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.763784    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.763896 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.763850    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/519960a2-3a77-4228-8f22-0455fcd3cfa9-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.764226 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:47.764206    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-serving-cert: secret "maas-api-e2e-extmod-a-660b59-serving-cert" not found
Oct 30 19:50:47.764460 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:50:47.764440    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls podName:519960a2-3a77-4228-8f22-0455fcd3cfa9 nodeName:}" failed. No retries permitted until 2026-10-30 19:50:48.264412314 +0000 UTC m=+3840.848895292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls") pod "maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh" (UID: "519960a2-3a77-4228-8f22-0455fcd3cfa9") : secret "maas-api-e2e-extmod-a-660b59-serving-cert" not found
Oct 30 19:50:47.765268 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.765243    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/519960a2-3a77-4228-8f22-0455fcd3cfa9-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.767121 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.767098    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-metrics-certs\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:47.823361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:47.823329    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq8j9\" (UniqueName: \"kubernetes.io/projected/519960a2-3a77-4228-8f22-0455fcd3cfa9-kube-api-access-kq8j9\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:48.269220 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:48.269183    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:48.271596 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:48.271570    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls\") pod \"maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:48.528962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:48.528873    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:48.904891 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:48.904651    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"]
Oct 30 19:50:48.909408 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:50:48.909378    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519960a2_3a77_4228_8f22_0455fcd3cfa9.slice/crio-4c68daf882c83035618101569172655c7354c4cb3a2d67ed4c4b4d915f6b8609 WatchSource:0}: Error finding container 4c68daf882c83035618101569172655c7354c4cb3a2d67ed4c4b4d915f6b8609: Status 404 returned error can't find the container with id 4c68daf882c83035618101569172655c7354c4cb3a2d67ed4c4b4d915f6b8609
Oct 30 19:50:49.798579 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:49.798541    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh" event={"ID":"519960a2-3a77-4228-8f22-0455fcd3cfa9","Type":"ContainerStarted","Data":"4c68daf882c83035618101569172655c7354c4cb3a2d67ed4c4b4d915f6b8609"}
Oct 30 19:50:50.806080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:50.806042    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh" event={"ID":"519960a2-3a77-4228-8f22-0455fcd3cfa9","Type":"ContainerStarted","Data":"0a9f4a21debab5b00195580d809d8c117ffeeaf4ebf11eaa08f1101131f18669"}
Oct 30 19:50:50.806533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:50.806140    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:50:50.928100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:50.928037    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh" podStartSLOduration=3.093016576 podStartE2EDuration="3.928019571s" totalImagesPullingTime="835.002995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:50.9245874 +0000 UTC m=+3843.509070397" watchObservedRunningTime="2026-10-30 19:50:50.928019571 +0000 UTC m=+3843.512502569"
Oct 30 19:50:56.830786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:50:56.830751    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:51:00.785225 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:00.785189    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"]
Oct 30 19:51:15.805740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:15.805695    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl" podUID="19d50bbe-1db0-41c2-8642-6618d5f1bff7" containerName="main" containerID="cri-o://5f849bb362a81dfb59444c279ec78fd68ddb84fa097a2efc142e5b78885453a5" gracePeriod=45
Oct 30 19:51:15.997412 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:15.997374    5232 generic.go:356] "Generic (PLEG): container finished" podID="19d50bbe-1db0-41c2-8642-6618d5f1bff7" containerID="5f849bb362a81dfb59444c279ec78fd68ddb84fa097a2efc142e5b78885453a5" exitCode=0
Oct 30 19:51:15.997590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:15.997424    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl" event={"ID":"19d50bbe-1db0-41c2-8642-6618d5f1bff7","Type":"ContainerDied","Data":"5f849bb362a81dfb59444c279ec78fd68ddb84fa097a2efc142e5b78885453a5"}
Oct 30 19:51:16.074054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.073991    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:51:16.188013 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:51:16.187973    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:51:16.216530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.216494    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-dshm\") pod \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") "
Oct 30 19:51:16.216698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.216574    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-model-cache\") pod \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") "
Oct 30 19:51:16.216698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.216617    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-home\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-home\") pod \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") "
Oct 30 19:51:16.216864 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.216759    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tmp-dir\") pod \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") "
Oct 30 19:51:16.216864 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.216805    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tls-certs\" (UniqueName: \"kubernetes.io/secret/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tls-certs\") pod \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") "
Oct 30 19:51:16.216989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.216898    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-model-cache" pod "19d50bbe-1db0-41c2-8642-6618d5f1bff7" (UID: "19d50bbe-1db0-41c2-8642-6618d5f1bff7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:16.216989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.216927    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19d50bbe-1db0-41c2-8642-6618d5f1bff7-kube-api-access-djd6s\" (UniqueName: \"kubernetes.io/projected/19d50bbe-1db0-41c2-8642-6618d5f1bff7-kube-api-access-djd6s\") pod \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\" (UID: \"19d50bbe-1db0-41c2-8642-6618d5f1bff7\") "
Oct 30 19:51:16.216989 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.216958    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-home" pod "19d50bbe-1db0-41c2-8642-6618d5f1bff7" (UID: "19d50bbe-1db0-41c2-8642-6618d5f1bff7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:16.217123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.217012    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tmp-dir" pod "19d50bbe-1db0-41c2-8642-6618d5f1bff7" (UID: "19d50bbe-1db0-41c2-8642-6618d5f1bff7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:16.217435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.217411    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:16.217538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.217438    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:16.217538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.217454    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:16.219129 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.219100    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-dshm" pod "19d50bbe-1db0-41c2-8642-6618d5f1bff7" (UID: "19d50bbe-1db0-41c2-8642-6618d5f1bff7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:16.219234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.219139    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d50bbe-1db0-41c2-8642-6618d5f1bff7-kube-api-access-djd6s" pod "19d50bbe-1db0-41c2-8642-6618d5f1bff7" (UID: "19d50bbe-1db0-41c2-8642-6618d5f1bff7"). InnerVolumeSpecName "kube-api-access-djd6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:16.219234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.219179    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tls-certs" pod "19d50bbe-1db0-41c2-8642-6618d5f1bff7" (UID: "19d50bbe-1db0-41c2-8642-6618d5f1bff7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:16.318357 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.318320    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d50bbe-1db0-41c2-8642-6618d5f1bff7-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:16.318357 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.318352    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djd6s\" (UniqueName: \"kubernetes.io/projected/19d50bbe-1db0-41c2-8642-6618d5f1bff7-kube-api-access-djd6s\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:16.318357 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:16.318365    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19d50bbe-1db0-41c2-8642-6618d5f1bff7-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:17.007806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:17.007758    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl" event={"ID":"19d50bbe-1db0-41c2-8642-6618d5f1bff7","Type":"ContainerDied","Data":"731265f0e0c9ab68fb03a7e626068571167eca11719ac0b1e81ca8eef0e7bcff"}
Oct 30 19:51:17.008233 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:17.007859    5232 scope.go:122] "RemoveContainer" containerID="5f849bb362a81dfb59444c279ec78fd68ddb84fa097a2efc142e5b78885453a5"
Oct 30 19:51:17.008233 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:17.008004    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"
Oct 30 19:51:17.180981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:17.180866    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"]
Oct 30 19:51:17.192919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:17.192886    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w1-fe7634/auth-test-model-fe7634-kserve-78646465d5-kcbtl"]
Oct 30 19:51:17.540576 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:17.540539    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19d50bbe-1db0-41c2-8642-6618d5f1bff7" path="/var/lib/kubelet/pods/19d50bbe-1db0-41c2-8642-6618d5f1bff7/volumes"
Oct 30 19:51:21.695548 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.695512    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"]
Oct 30 19:51:21.696140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.696110    5232 cpu_manager.go:414] "containerMap: removing container" podUID="19d50bbe-1db0-41c2-8642-6618d5f1bff7" containerName="main"
Oct 30 19:51:21.696140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.696126    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19d50bbe-1db0-41c2-8642-6618d5f1bff7" containerName="main"
Oct 30 19:51:21.696263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.696251    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="19d50bbe-1db0-41c2-8642-6618d5f1bff7" containerName="main"
Oct 30 19:51:21.699678 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.699658    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.750093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.750063    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-905752-openshift-default-dockercfg-cxcdk\""
Oct 30 19:51:21.803240 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.803194    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"]
Oct 30 19:51:21.813223 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813187    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-certs\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.813404 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813245    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-socket\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.813404 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813276    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-credential-socket\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.813404 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813322    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-podinfo\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.813578 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813403    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-envoy\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.813578 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813437    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xgbp\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-kube-api-access-2xgbp\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.813578 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813480    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-token\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.813578 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813563    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-data\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.813791 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.813608    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istiod-ca-cert\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.914498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914453    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-envoy\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.914498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914501    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xgbp\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-kube-api-access-2xgbp\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.914765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914540    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-token\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.914765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914630    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-data\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.914765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914682    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istiod-ca-cert\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.914765 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914758    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-certs\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.915104 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914786    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-socket\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.915104 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914806    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-credential-socket\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.915104 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.914863    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-podinfo\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.915317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.915095    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-data\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.915317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.915272    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-socket\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.915317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.915294    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-credential-socket\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.915424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.915321    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-certs\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.915639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.915619    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istiod-ca-cert\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.916783 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.916761    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-envoy\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.917180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.917160    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-podinfo\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.975252 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.975178    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-token\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:21.975728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:21.975255    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xgbp\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-kube-api-access-2xgbp\") pod \"e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") " pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:22.012795 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:22.012759    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:22.459416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:22.459383    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"]
Oct 30 19:51:22.464127 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:51:22.464099    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14d8f134_a0b0_4bb3_8f12_361e41b1e5d4.slice/crio-43e5bbacecc41ff619f86d33c9dcc352e31028a36692315d765983cf38c8f235 WatchSource:0}: Error finding container 43e5bbacecc41ff619f86d33c9dcc352e31028a36692315d765983cf38c8f235: Status 404 returned error can't find the container with id 43e5bbacecc41ff619f86d33c9dcc352e31028a36692315d765983cf38c8f235
Oct 30 19:51:22.466679 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:22.466646    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:22.466754 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:22.466717    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:22.466754 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:22.466748    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:23.051944 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:23.051896    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" event={"ID":"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4","Type":"ContainerStarted","Data":"07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2"}
Oct 30 19:51:23.052462 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:23.052007    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" event={"ID":"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4","Type":"ContainerStarted","Data":"43e5bbacecc41ff619f86d33c9dcc352e31028a36692315d765983cf38c8f235"}
Oct 30 19:51:23.174062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:23.173997    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" podStartSLOduration=2.17398155 podStartE2EDuration="2.17398155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:23.172568905 +0000 UTC m=+3875.757051902" watchObservedRunningTime="2026-10-30 19:51:23.17398155 +0000 UTC m=+3875.758464547"
Oct 30 19:51:24.013678 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:24.013639    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:24.019310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:24.019284    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:24.059011 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:24.058981    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:24.059906 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:24.059887    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:51:46.188385 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:51:46.188306    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:51:48.152299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:48.152267    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:51:48.157170 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:48.157139    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:51:48.176669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:48.176644    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:51:48.180709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:48.180687    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:51:49.603598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.603566    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"]
Oct 30 19:51:49.609573 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.609554    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.660015 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.659984    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-b330be-serving-cert\""
Oct 30 19:51:49.710589 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.710552    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"]
Oct 30 19:51:49.749558 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.749520    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qphcw\" (UniqueName: \"kubernetes.io/projected/1a60656a-6b08-4198-9649-23c3e07c4399-kube-api-access-qphcw\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.749735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.749606    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-maas-api-tls\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.749735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.749707    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-metrics-certs\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.749925 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.749904    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a60656a-6b08-4198-9649-23c3e07c4399-openshift-service-ca\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.851519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.851481    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a60656a-6b08-4198-9649-23c3e07c4399-openshift-service-ca\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.851707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.851593    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qphcw\" (UniqueName: \"kubernetes.io/projected/1a60656a-6b08-4198-9649-23c3e07c4399-kube-api-access-qphcw\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.851707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.851612    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-maas-api-tls\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.851707 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.851645    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-metrics-certs\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.852242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.852217    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a60656a-6b08-4198-9649-23c3e07c4399-openshift-service-ca\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.854142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.854092    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-maas-api-tls\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.854236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.854152    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-metrics-certs\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.911209 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.911172    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qphcw\" (UniqueName: \"kubernetes.io/projected/1a60656a-6b08-4198-9649-23c3e07c4399-kube-api-access-qphcw\") pod \"maas-api-e2e-explicit-b330be-64797bf769-b6vcv\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:49.924115 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:49.924082    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:50.302628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:50.302375    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"]
Oct 30 19:51:50.306566 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:51:50.306537    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a60656a_6b08_4198_9649_23c3e07c4399.slice/crio-d00b4c01ac6b70e9b2d2fdeacdfe1269654988c4a284f5fd4c11776df1f9cd56 WatchSource:0}: Error finding container d00b4c01ac6b70e9b2d2fdeacdfe1269654988c4a284f5fd4c11776df1f9cd56: Status 404 returned error can't find the container with id d00b4c01ac6b70e9b2d2fdeacdfe1269654988c4a284f5fd4c11776df1f9cd56
Oct 30 19:51:50.307973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:50.307953    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:51:51.237149 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:51.237115    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv" event={"ID":"1a60656a-6b08-4198-9649-23c3e07c4399","Type":"ContainerStarted","Data":"d00b4c01ac6b70e9b2d2fdeacdfe1269654988c4a284f5fd4c11776df1f9cd56"}
Oct 30 19:51:52.245237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:52.245187    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv" event={"ID":"1a60656a-6b08-4198-9649-23c3e07c4399","Type":"ContainerStarted","Data":"4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157"}
Oct 30 19:51:52.245748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:52.245353    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:51:52.360068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:52.360008    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv" podStartSLOduration=2.437954522 podStartE2EDuration="3.359991278s" totalImagesPullingTime="922.036756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:52.358885382 +0000 UTC m=+3904.943368379" watchObservedRunningTime="2026-10-30 19:51:52.359991278 +0000 UTC m=+3904.944474277"
Oct 30 19:51:58.257453 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:51:58.257424    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:52:16.202509 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:52:16.202465    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:52:16.669839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:16.669784    5232 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:52:16.670054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:16.669867    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 19:52:17.457110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:17.453943    5232 generic.go:356] "Generic (PLEG): container finished" podID="098d5771-830f-45a6-acda-76ca9d7dd365" containerID="c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2" exitCode=0
Oct 30 19:52:17.457110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:17.454139    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" event={"ID":"098d5771-830f-45a6-acda-76ca9d7dd365","Type":"ContainerDied","Data":"c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2"}
Oct 30 19:52:17.457110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:17.454652    5232 scope.go:122] "RemoveContainer" containerID="c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2"
Oct 30 19:52:17.458127 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:17.458087    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:17.459093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:17.458195    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:17.459093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:17.458249    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:18.469499 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:18.469463    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" event={"ID":"098d5771-830f-45a6-acda-76ca9d7dd365","Type":"ContainerStarted","Data":"53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83"}
Oct 30 19:52:18.581424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:18.581381    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:52:18.581650 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:18.581601    5232 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:52:18.581746 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:18.581684    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 19:52:19.581640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:19.581596    5232 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:52:19.582182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:19.581668    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 19:52:20.581345 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:20.581305    5232 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:52:20.581556 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:20.581368    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 19:52:21.581808 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:21.581768    5232 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:52:21.582319 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:21.581857    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 19:52:22.582150 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:22.582117    5232 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:52:22.582572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:22.582174    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 19:52:23.582085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:23.582047    5232 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:52:23.582276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:23.582132    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.169:15021/healthz/ready\": dial tcp 10.133.0.169:15021: connect: connection refused"
Oct 30 19:52:24.587204 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:24.587173    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:52:24.587720 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:24.587471    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:52:24.588262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:24.588239    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:52:25.541553 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.532744    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"]
Oct 30 19:52:25.542980 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.542933    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.600802 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.600651    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"]
Oct 30 19:52:25.643244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643212    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istiod-ca-cert\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.643406 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643251    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-socket\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.643449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643399    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js7c4\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-kube-api-access-js7c4\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.643503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643488    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-podinfo\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.643622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643610    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-envoy\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.643681 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643667    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-certs\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.643774 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643731    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-data\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.643906 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643887    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-credential-socket\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.644045 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.643917    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-token\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.644489 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.644472    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-312161-openshift-default-dockercfg-kcdv9\""
Oct 30 19:52:25.745009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.744971    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-certs\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745047    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-data\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745100    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-credential-socket\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745214 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745200    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-token\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745272    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istiod-ca-cert\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745306    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-socket\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745423    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-certs\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745469    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js7c4\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-kube-api-access-js7c4\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745514    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-podinfo\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745540    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-envoy\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.745773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.745557    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-data\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.746093 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.746068    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istiod-ca-cert\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.746297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.746272    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-socket\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.746371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.746299    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-credential-socket\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.748168 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.748144    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-envoy\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.748311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.748289    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-podinfo\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.804435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.804361    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-token\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.804435 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.804427    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js7c4\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-kube-api-access-js7c4\") pod \"e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") " pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:25.861685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:25.861646    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:26.291081 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:26.291045    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"]
Oct 30 19:52:26.295548 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:52:26.295522    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50ac16c_f6e9_42f5_8e35_2cab4009e3bb.slice/crio-cb7a91794f16f9fb86b04d8d931d0023bc738d1ee405c7e7fb696a89769b8aaa WatchSource:0}: Error finding container cb7a91794f16f9fb86b04d8d931d0023bc738d1ee405c7e7fb696a89769b8aaa: Status 404 returned error can't find the container with id cb7a91794f16f9fb86b04d8d931d0023bc738d1ee405c7e7fb696a89769b8aaa
Oct 30 19:52:26.298149 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:26.298113    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:26.298258 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:26.298194    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:26.298324 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:26.298263    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:26.543742 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:26.536202    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" event={"ID":"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb","Type":"ContainerStarted","Data":"cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f"}
Oct 30 19:52:26.543742 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:26.536258    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" event={"ID":"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb","Type":"ContainerStarted","Data":"cb7a91794f16f9fb86b04d8d931d0023bc738d1ee405c7e7fb696a89769b8aaa"}
Oct 30 19:52:26.655698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:26.655631    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podStartSLOduration=1.6556161839999999 podStartE2EDuration="1.655616184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:26.652860881 +0000 UTC m=+3939.237343878" watchObservedRunningTime="2026-10-30 19:52:26.655616184 +0000 UTC m=+3939.240099180"
Oct 30 19:52:26.862738 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:26.862701    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:27.868613 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:27.868582    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:28.549384 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:28.549348    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:28.550240 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:28.550221    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:52:46.190797 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:52:46.190760    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:52:51.722297 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.722256    5232 generic.go:356] "Generic (PLEG): container finished" podID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerID="c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6" exitCode=0
Oct 30 19:52:51.722734 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.722332    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" event={"ID":"494821ca-b40d-4423-b192-0ce5a101eeb4","Type":"ContainerDied","Data":"c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6"}
Oct 30 19:52:51.722785 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.722753    5232 scope.go:122] "RemoveContainer" containerID="c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6"
Oct 30 19:52:51.724373 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.724341    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:51.724496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.724450    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:51.724564 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.724514    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:51.829250 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.829213    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"]
Oct 30 19:52:51.835088 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.835009    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:51.887404 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.887364    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-905752-serving-cert\""
Oct 30 19:52:51.891424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.891390    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"]
Oct 30 19:52:51.909638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.909598    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:51.909805 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.909762    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb2hm\" (UniqueName: \"kubernetes.io/projected/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-kube-api-access-wb2hm\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:51.909914 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.909893    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-metrics-certs\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:51.909969 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:51.909923    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.011883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.011836    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.012085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.011949    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb2hm\" (UniqueName: \"kubernetes.io/projected/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-kube-api-access-wb2hm\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.012085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.012028    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-metrics-certs\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.012085 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.012054    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.012314 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:52:52.012297    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-serving-cert: secret "maas-api-e2e-extmod-a-905752-serving-cert" not found
Oct 30 19:52:52.012390 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:52:52.012376    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls podName:4a517ee6-c6b5-4350-9823-1c3fddfd75c2 nodeName:}" failed. No retries permitted until 2026-10-30 19:52:52.512356069 +0000 UTC m=+3965.096839051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls") pod "maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8" (UID: "4a517ee6-c6b5-4350-9823-1c3fddfd75c2") : secret "maas-api-e2e-extmod-a-905752-serving-cert" not found
Oct 30 19:52:52.013567 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.013342    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.015083 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.015062    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-metrics-certs\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.071575 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.071541    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb2hm\" (UniqueName: \"kubernetes.io/projected/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-kube-api-access-wb2hm\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.518403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.518367    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.521241 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.521211    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls\") pod \"maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.730571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.730536    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" event={"ID":"494821ca-b40d-4423-b192-0ce5a101eeb4","Type":"ContainerStarted","Data":"6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9"}
Oct 30 19:52:52.759850 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.759058    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:52.884289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.881008    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:52:52.884289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.882731    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:52:52.884289 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:52.882783    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:52:53.164068 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:53.164039    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"]
Oct 30 19:52:53.166606 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:52:53.166574    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a517ee6_c6b5_4350_9823_1c3fddfd75c2.slice/crio-114aeaab533205c820b0b38f356e4024f33d6c0dd972d3dd027f7cb877eeb940 WatchSource:0}: Error finding container 114aeaab533205c820b0b38f356e4024f33d6c0dd972d3dd027f7cb877eeb940: Status 404 returned error can't find the container with id 114aeaab533205c820b0b38f356e4024f33d6c0dd972d3dd027f7cb877eeb940
Oct 30 19:52:53.765845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:53.765773    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8" event={"ID":"4a517ee6-c6b5-4350-9823-1c3fddfd75c2","Type":"ContainerStarted","Data":"114aeaab533205c820b0b38f356e4024f33d6c0dd972d3dd027f7cb877eeb940"}
Oct 30 19:52:53.881594 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:53.881562    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:52:53.881781 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:53.881616    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:52:54.791836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:54.790720    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8" event={"ID":"4a517ee6-c6b5-4350-9823-1c3fddfd75c2","Type":"ContainerStarted","Data":"2aa6f18d8157e819a158534dd159876472204aac47e3b4e58021fc07aea83d69"}
Oct 30 19:52:54.791836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:54.791799    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:52:54.881248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:54.881206    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:52:54.881427 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:54.881274    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:52:54.912102 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:54.912046    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8" podStartSLOduration=2.979812105 podStartE2EDuration="3.91203155s" totalImagesPullingTime="932.219445ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:54.909622156 +0000 UTC m=+3967.494105166" watchObservedRunningTime="2026-10-30 19:52:54.91203155 +0000 UTC m=+3967.496514547"
Oct 30 19:52:55.880829 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:55.880783    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:52:55.881337 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:55.880864    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:52:56.746836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:56.740892    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:52:56.880806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:56.880747    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:52:56.880984 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:56.880868    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:52:57.881181 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:57.881147    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:52:57.881568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:57.881202    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:52:58.885350 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:58.885318    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:52:58.886426 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:52:58.886406    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:53:01.809663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:01.809631    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:53:16.190009 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:53:16.189968    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:53:21.002627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:21.002593    5232 generic.go:356] "Generic (PLEG): container finished" podID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerID="07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2" exitCode=0
Oct 30 19:53:21.003205 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:21.002672    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" event={"ID":"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4","Type":"ContainerDied","Data":"07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2"}
Oct 30 19:53:21.003280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:21.003249    5232 scope.go:122] "RemoveContainer" containerID="07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2"
Oct 30 19:53:21.005743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:21.005707    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:21.005863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:21.005782    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:21.005863 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:21.005849    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:22.013134 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:22.013092    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" event={"ID":"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4","Type":"ContainerStarted","Data":"c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6"}
Oct 30 19:53:23.013378 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:23.013341    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:53:23.017882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:23.017851    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:53:23.018109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:23.017926    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 19:53:24.013096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:24.013063    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:53:24.013293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:24.013127    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 19:53:24.061976 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:24.061944    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:53:25.013599 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:25.013553    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:53:25.013844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:25.013675    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 19:53:26.013981 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:26.013945    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:53:26.014396 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:26.014009    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 19:53:27.014054 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:27.014021    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:53:27.014566 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:27.014075    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 19:53:28.018034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:28.017997    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:53:28.019177 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:53:28.019153    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:53:46.188037 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:53:46.187995    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:54:03.789494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.789456    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"]
Oct 30 19:54:03.792947 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.792930    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.843177 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.843145    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-312161-serving-cert\""
Oct 30 19:54:03.882845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.882794    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-metrics-certs\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.883051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.882852    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn8cb\" (UniqueName: \"kubernetes.io/projected/330a6418-9381-4c85-a56c-5e238567f9f0-kube-api-access-xn8cb\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.883051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.883005    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.883051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.883035    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/330a6418-9381-4c85-a56c-5e238567f9f0-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.894158 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.894123    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"]
Oct 30 19:54:03.984344 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.984304    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-metrics-certs\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.984538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.984348    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn8cb\" (UniqueName: \"kubernetes.io/projected/330a6418-9381-4c85-a56c-5e238567f9f0-kube-api-access-xn8cb\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.984538 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.984522    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.984627 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.984573    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/330a6418-9381-4c85-a56c-5e238567f9f0-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.984677 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:54:03.984654    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-serving-cert: secret "maas-api-e2e-extmod-b-312161-serving-cert" not found
Oct 30 19:54:03.984764 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:54:03.984750    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls podName:330a6418-9381-4c85-a56c-5e238567f9f0 nodeName:}" failed. No retries permitted until 2026-10-30 19:54:04.484732712 +0000 UTC m=+4037.069215688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls") pod "maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr" (UID: "330a6418-9381-4c85-a56c-5e238567f9f0") : secret "maas-api-e2e-extmod-b-312161-serving-cert" not found
Oct 30 19:54:03.985294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.985271    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/330a6418-9381-4c85-a56c-5e238567f9f0-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:03.986885 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:03.986860    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-metrics-certs\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:04.044230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:04.044154    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn8cb\" (UniqueName: \"kubernetes.io/projected/330a6418-9381-4c85-a56c-5e238567f9f0-kube-api-access-xn8cb\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:04.492080 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:04.492044    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:04.494660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:04.494630    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls\") pod \"maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:04.705711 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:04.705675    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:05.083547 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:05.083320    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"]
Oct 30 19:54:05.087840 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:54:05.087799    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod330a6418_9381_4c85_a56c_5e238567f9f0.slice/crio-45f724f2c41d401b87368216a03889effd2d0f5badae1faadec1dac772f7882e WatchSource:0}: Error finding container 45f724f2c41d401b87368216a03889effd2d0f5badae1faadec1dac772f7882e: Status 404 returned error can't find the container with id 45f724f2c41d401b87368216a03889effd2d0f5badae1faadec1dac772f7882e
Oct 30 19:54:05.331011 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:05.330967    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr" event={"ID":"330a6418-9381-4c85-a56c-5e238567f9f0","Type":"ContainerStarted","Data":"45f724f2c41d401b87368216a03889effd2d0f5badae1faadec1dac772f7882e"}
Oct 30 19:54:06.339659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:06.339625    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr" event={"ID":"330a6418-9381-4c85-a56c-5e238567f9f0","Type":"ContainerStarted","Data":"5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803"}
Oct 30 19:54:06.340108 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:06.339693    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:06.455184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:06.455134    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr" podStartSLOduration=2.589769085 podStartE2EDuration="3.455120432s" totalImagesPullingTime="865.351347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:54:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:54:06.45439759 +0000 UTC m=+4039.038880584" watchObservedRunningTime="2026-10-30 19:54:06.455120432 +0000 UTC m=+4039.039603473"
Oct 30 19:54:12.351470 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:12.351440    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:54:16.187970 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:54:16.187932    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:54:29.591995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:29.591949    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:54:29.592402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:29.592025    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:54:30.036869 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.035311    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"]
Oct 30 19:54:30.042290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.042264    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.121705 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.121666    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"]
Oct 30 19:54:30.158639 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.158593    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.158857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.158662    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.158857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.158721    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.158857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.158841    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.159077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.158916    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5jk8\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-kube-api-access-x5jk8\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.159077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.158967    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.159077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.159011    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.159077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.159049    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.159311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.159100    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5401cf32-a2fd-4804-b703-5cf2116c6510-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.162275 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.162247    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-7b6aee-openshift-default-dockercfg-n2tpq\""
Oct 30 19:54:30.260555 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260520    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.260735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260564    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5jk8\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-kube-api-access-x5jk8\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.260735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260619    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.260735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260657    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.260735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260690    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260735    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5401cf32-a2fd-4804-b703-5cf2116c6510-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260883    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260951    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261009 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.260996    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.261048    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.261174    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.261288    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261413 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.261398    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.261698 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.261669    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5401cf32-a2fd-4804-b703-5cf2116c6510-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.263190 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.263168    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.263418 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.263399    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.320152 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.320076    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.320281 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.320210    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5jk8\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-kube-api-access-x5jk8\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.359614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.359581    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:30.794920 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.792877    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"]
Oct 30 19:54:30.802224 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:54:30.799870    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5401cf32_a2fd_4804_b703_5cf2116c6510.slice/crio-6680a1a65f38dc2b78d7ff331b33f81bd278c49612f6d9065ef5e820ae7dca41 WatchSource:0}: Error finding container 6680a1a65f38dc2b78d7ff331b33f81bd278c49612f6d9065ef5e820ae7dca41: Status 404 returned error can't find the container with id 6680a1a65f38dc2b78d7ff331b33f81bd278c49612f6d9065ef5e820ae7dca41
Oct 30 19:54:30.802884 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.802846    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:30.802964 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.802920    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:30.802964 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:30.802949    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:31.556843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.556717    5232 generic.go:356] "Generic (PLEG): container finished" podID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerID="cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f" exitCode=0
Oct 30 19:54:31.557451 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.556900    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" event={"ID":"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb","Type":"ContainerDied","Data":"cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f"}
Oct 30 19:54:31.558123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.558103    5232 scope.go:122] "RemoveContainer" containerID="cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f"
Oct 30 19:54:31.562403 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.562374    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" event={"ID":"5401cf32-a2fd-4804-b703-5cf2116c6510","Type":"ContainerStarted","Data":"bfce85e8cda32be67cc186faf3b4a0f22d70d7aba0e7d1cc93a04a842cc5457f"}
Oct 30 19:54:31.562583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.562564    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" event={"ID":"5401cf32-a2fd-4804-b703-5cf2116c6510","Type":"ContainerStarted","Data":"6680a1a65f38dc2b78d7ff331b33f81bd278c49612f6d9065ef5e820ae7dca41"}
Oct 30 19:54:31.565266 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.565228    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:31.565480 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.565457    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:31.565628 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.565610    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:31.795905 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:31.795803    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" podStartSLOduration=2.795786209 podStartE2EDuration="2.795786209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:54:31.795020372 +0000 UTC m=+4064.379503369" watchObservedRunningTime="2026-10-30 19:54:31.795786209 +0000 UTC m=+4064.380269267"
Oct 30 19:54:32.367838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:32.367186    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:32.367838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:32.367361    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:54:32.367838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:32.367405    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 19:54:32.595945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:32.595643    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" event={"ID":"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb","Type":"ContainerStarted","Data":"617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314"}
Oct 30 19:54:32.870269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:32.870231    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:54:32.871005 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:32.870914    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 19:54:32.871196 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:32.871170    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 19:54:33.364371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:33.364328    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:54:33.364704 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:33.364659    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 19:54:33.862654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:33.862198    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 19:54:33.862654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:33.862261    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 19:54:34.360245 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:34.360204    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:54:34.360904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:34.360870    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 19:54:34.865838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:34.865209    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 19:54:34.865838 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:34.865278    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 19:54:35.362837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:35.362474    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:54:35.362837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:35.362538    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 19:54:35.866322 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:35.866282    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 19:54:35.866659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:35.866605    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 19:54:36.363837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:36.363198    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:54:36.363837 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:36.363268    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 19:54:36.864288 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:36.864252    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 19:54:36.864610 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:36.864571    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 19:54:37.379170 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:37.379138    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:37.380402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:37.380332    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:37.381071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:37.381033    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:54:37.862894 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:37.862854    5232 patch_prober.go:29] interesting pod/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 19:54:37.863121 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:37.862910    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 19:54:38.867426 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:38.867397    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:54:38.867871 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:38.867832    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:54:38.868609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:38.868592    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:54:46.190977 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:54:46.187697    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:54:55.753571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:55.753533    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"]
Oct 30 19:54:55.754089 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:55.753782    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf" podUID="5ba1a1aa-85f3-4750-ba11-a3ac1b686779" containerName="maas-api" containerID="cri-o://f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2" gracePeriod=30
Oct 30 19:54:56.032200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.032165    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:54:56.179039 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.178995    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls\") pod \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") "
Oct 30 19:54:56.179236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.179194    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-metrics-certs\") pod \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") "
Oct 30 19:54:56.179313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.179249    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-openshift-service-ca\") pod \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") "
Oct 30 19:54:56.179313 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.179297    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-kube-api-access-s2dq4\" (UniqueName: \"kubernetes.io/projected/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-kube-api-access-s2dq4\") pod \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\" (UID: \"5ba1a1aa-85f3-4750-ba11-a3ac1b686779\") "
Oct 30 19:54:56.179663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.179635    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-openshift-service-ca" pod "5ba1a1aa-85f3-4750-ba11-a3ac1b686779" (UID: "5ba1a1aa-85f3-4750-ba11-a3ac1b686779"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:56.180043 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.180020    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:56.181450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.181412    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls" pod "5ba1a1aa-85f3-4750-ba11-a3ac1b686779" (UID: "5ba1a1aa-85f3-4750-ba11-a3ac1b686779"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:54:56.181622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.181596    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-kube-api-access-s2dq4" pod "5ba1a1aa-85f3-4750-ba11-a3ac1b686779" (UID: "5ba1a1aa-85f3-4750-ba11-a3ac1b686779"). InnerVolumeSpecName "kube-api-access-s2dq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:56.181755 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.181736    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-metrics-certs" pod "5ba1a1aa-85f3-4750-ba11-a3ac1b686779" (UID: "5ba1a1aa-85f3-4750-ba11-a3ac1b686779"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:54:56.281604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.281503    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:56.281604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.281551    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2dq4\" (UniqueName: \"kubernetes.io/projected/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-kube-api-access-s2dq4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:56.281604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.281569    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba1a1aa-85f3-4750-ba11-a3ac1b686779-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:56.772667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.772635    5232 generic.go:356] "Generic (PLEG): container finished" podID="5ba1a1aa-85f3-4750-ba11-a3ac1b686779" containerID="f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2" exitCode=0
Oct 30 19:54:56.773206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.772683    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"
Oct 30 19:54:56.773206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.772726    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf" event={"ID":"5ba1a1aa-85f3-4750-ba11-a3ac1b686779","Type":"ContainerDied","Data":"f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2"}
Oct 30 19:54:56.773206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.772787    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf" event={"ID":"5ba1a1aa-85f3-4750-ba11-a3ac1b686779","Type":"ContainerDied","Data":"fae7f11eb12fdffd6611fb01415ea2ce77dbf8d45a9a121cdf6a0f0e25e250ad"}
Oct 30 19:54:56.773206 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.772825    5232 scope.go:122] "RemoveContainer" containerID="f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2"
Oct 30 19:54:56.783954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.783933    5232 scope.go:122] "RemoveContainer" containerID="f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2"
Oct 30 19:54:56.784225 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:54:56.784204    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2\": container with ID starting with f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2 not found: ID does not exist" containerID="f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2"
Oct 30 19:54:56.784310 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.784231    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2"} err="failed to get container status \"f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2\": rpc error: code = NotFound desc = could not find container \"f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2\": container with ID starting with f1963f9e106629bdd9ca09756cc9a86e4e3f3f0236ac4f040a959c97e07833a2 not found: ID does not exist"
Oct 30 19:54:56.876523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.876486    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"]
Oct 30 19:54:56.876769 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.876726    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv" podUID="1a60656a-6b08-4198-9649-23c3e07c4399" containerName="maas-api" containerID="cri-o://4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157" gracePeriod=30
Oct 30 19:54:56.933703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.933671    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"]
Oct 30 19:54:56.939442 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:56.939412    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-17216f-579db46d64-bt2hf"]
Oct 30 19:54:57.162671 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.162641    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:54:57.304729 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.304689    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a60656a-6b08-4198-9649-23c3e07c4399-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a60656a-6b08-4198-9649-23c3e07c4399-openshift-service-ca\") pod \"1a60656a-6b08-4198-9649-23c3e07c4399\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") "
Oct 30 19:54:57.304945 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.304870    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a60656a-6b08-4198-9649-23c3e07c4399-kube-api-access-qphcw\" (UniqueName: \"kubernetes.io/projected/1a60656a-6b08-4198-9649-23c3e07c4399-kube-api-access-qphcw\") pod \"1a60656a-6b08-4198-9649-23c3e07c4399\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") "
Oct 30 19:54:57.305022 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.304947    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-metrics-certs\") pod \"1a60656a-6b08-4198-9649-23c3e07c4399\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") "
Oct 30 19:54:57.305136 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.305108    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a60656a-6b08-4198-9649-23c3e07c4399-openshift-service-ca" pod "1a60656a-6b08-4198-9649-23c3e07c4399" (UID: "1a60656a-6b08-4198-9649-23c3e07c4399"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:57.305231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.305134    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-maas-api-tls\") pod \"1a60656a-6b08-4198-9649-23c3e07c4399\" (UID: \"1a60656a-6b08-4198-9649-23c3e07c4399\") "
Oct 30 19:54:57.305719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.305685    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a60656a-6b08-4198-9649-23c3e07c4399-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:57.308352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.308278    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-maas-api-tls" pod "1a60656a-6b08-4198-9649-23c3e07c4399" (UID: "1a60656a-6b08-4198-9649-23c3e07c4399"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:54:57.308696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.308661    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a60656a-6b08-4198-9649-23c3e07c4399-kube-api-access-qphcw" pod "1a60656a-6b08-4198-9649-23c3e07c4399" (UID: "1a60656a-6b08-4198-9649-23c3e07c4399"). InnerVolumeSpecName "kube-api-access-qphcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:57.309318 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.309297    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-metrics-certs" pod "1a60656a-6b08-4198-9649-23c3e07c4399" (UID: "1a60656a-6b08-4198-9649-23c3e07c4399"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:54:57.407370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.407327    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qphcw\" (UniqueName: \"kubernetes.io/projected/1a60656a-6b08-4198-9649-23c3e07c4399-kube-api-access-qphcw\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:57.407370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.407374    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:57.407602 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.407391    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a60656a-6b08-4198-9649-23c3e07c4399-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:57.550053 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.550007    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ba1a1aa-85f3-4750-ba11-a3ac1b686779" path="/var/lib/kubelet/pods/5ba1a1aa-85f3-4750-ba11-a3ac1b686779/volumes"
Oct 30 19:54:57.793924 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.793887    5232 generic.go:356] "Generic (PLEG): container finished" podID="1a60656a-6b08-4198-9649-23c3e07c4399" containerID="4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157" exitCode=0
Oct 30 19:54:57.794696 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.794669    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv" event={"ID":"1a60656a-6b08-4198-9649-23c3e07c4399","Type":"ContainerDied","Data":"4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157"}
Oct 30 19:54:57.794876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.794860    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv" event={"ID":"1a60656a-6b08-4198-9649-23c3e07c4399","Type":"ContainerDied","Data":"d00b4c01ac6b70e9b2d2fdeacdfe1269654988c4a284f5fd4c11776df1f9cd56"}
Oct 30 19:54:57.795004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.794992    5232 scope.go:122] "RemoveContainer" containerID="4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157"
Oct 30 19:54:57.795262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.795248    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"
Oct 30 19:54:57.821709 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.821682    5232 scope.go:122] "RemoveContainer" containerID="4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157"
Oct 30 19:54:57.822728 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:54:57.822051    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157\": container with ID starting with 4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157 not found: ID does not exist" containerID="4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157"
Oct 30 19:54:57.822728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.822087    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157"} err="failed to get container status \"4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157\": rpc error: code = NotFound desc = could not find container \"4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157\": container with ID starting with 4ecd2c17a4cfdc5463aada8c81fd161a6d782b00578b19e406b982e02993b157 not found: ID does not exist"
Oct 30 19:54:57.969919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:57.969883    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"]
Oct 30 19:54:58.017444 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:58.017406    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b330be-64797bf769-b6vcv"]
Oct 30 19:54:59.540290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:54:59.540253    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a60656a-6b08-4198-9649-23c3e07c4399" path="/var/lib/kubelet/pods/1a60656a-6b08-4198-9649-23c3e07c4399/volumes"
Oct 30 19:55:06.926363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:06.926309    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"]
Oct 30 19:55:06.926856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:06.926611    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" containerID="cri-o://53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83" gracePeriod=30
Oct 30 19:55:09.607461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:09.607299    5232 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:55:09.607461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:09.607367    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:55:12.195342 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.195276    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:55:12.290949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.290915    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-istio-token\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-istio-token\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.290949 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.290951    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/098d5771-830f-45a6-acda-76ca9d7dd365-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/098d5771-830f-45a6-acda-76ca9d7dd365-istio-podinfo\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.291184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291008    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-data\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.291184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291038    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-credential-socket\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.291184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291120    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/098d5771-830f-45a6-acda-76ca9d7dd365-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/098d5771-830f-45a6-acda-76ca9d7dd365-istiod-ca-cert\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.291184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291156    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-socket\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.291414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291252    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-envoy\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.291414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291296    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-certs\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.291414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291371    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-kube-api-access-9pths\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-kube-api-access-9pths\") pod \"098d5771-830f-45a6-acda-76ca9d7dd365\" (UID: \"098d5771-830f-45a6-acda-76ca9d7dd365\") "
Oct 30 19:55:12.291414 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291369    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-data" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.291700 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291439    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-credential-socket" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.291700 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291583    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-certs" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.291700 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291614    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/098d5771-830f-45a6-acda-76ca9d7dd365-istiod-ca-cert" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:55:12.291931 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291725    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-socket" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.292004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.291980    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/098d5771-830f-45a6-acda-76ca9d7dd365-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.292076 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.292011    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.292076 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.292027    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.292076 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.292043    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.292076 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.292059    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.293395 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.293364    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-istio-token" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:12.293532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.293460    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-envoy" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.293532 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.293514    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/098d5771-830f-45a6-acda-76ca9d7dd365-istio-podinfo" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:55:12.293654 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.293628    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-kube-api-access-9pths" pod "098d5771-830f-45a6-acda-76ca9d7dd365" (UID: "098d5771-830f-45a6-acda-76ca9d7dd365"). InnerVolumeSpecName "kube-api-access-9pths". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:12.393371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.393331    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.393371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.393362    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/098d5771-830f-45a6-acda-76ca9d7dd365-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.393371 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.393374    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/098d5771-830f-45a6-acda-76ca9d7dd365-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.393685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.393383    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pths\" (UniqueName: \"kubernetes.io/projected/098d5771-830f-45a6-acda-76ca9d7dd365-kube-api-access-9pths\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.914182 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.914140    5232 generic.go:356] "Generic (PLEG): container finished" podID="098d5771-830f-45a6-acda-76ca9d7dd365" containerID="53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83" exitCode=0
Oct 30 19:55:12.914372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.914184    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" event={"ID":"098d5771-830f-45a6-acda-76ca9d7dd365","Type":"ContainerDied","Data":"53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83"}
Oct 30 19:55:12.914372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.914210    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"
Oct 30 19:55:12.914372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.914242    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4" event={"ID":"098d5771-830f-45a6-acda-76ca9d7dd365","Type":"ContainerDied","Data":"84bec3062b545433054947d5175618c7a3b04c4e50f455ae19927461ebb07044"}
Oct 30 19:55:12.914372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.914264    5232 scope.go:122] "RemoveContainer" containerID="53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83"
Oct 30 19:55:12.928739 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.928711    5232 scope.go:122] "RemoveContainer" containerID="c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2"
Oct 30 19:55:12.938484 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.938462    5232 scope.go:122] "RemoveContainer" containerID="53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83"
Oct 30 19:55:12.938744 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:55:12.938722    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83\": container with ID starting with 53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83 not found: ID does not exist" containerID="53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83"
Oct 30 19:55:12.938845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.938757    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83"} err="failed to get container status \"53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83\": rpc error: code = NotFound desc = could not find container \"53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83\": container with ID starting with 53ac1cee178b5a29597988709279a26f738bbf8803f2b59351bc4481d0850c83 not found: ID does not exist"
Oct 30 19:55:12.938845 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.938785    5232 scope.go:122] "RemoveContainer" containerID="c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2"
Oct 30 19:55:12.939091 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:55:12.939052    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2\": container with ID starting with c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2 not found: ID does not exist" containerID="c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2"
Oct 30 19:55:12.939140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:12.939098    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2"} err="failed to get container status \"c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2\": rpc error: code = NotFound desc = could not find container \"c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2\": container with ID starting with c4e81b792cf65c53dfeea4c1b52662094a44b6c6d95764e23baf48fc1cba2cf2 not found: ID does not exist"
Oct 30 19:55:13.074594 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:13.074559    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"]
Oct 30 19:55:13.089655 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:13.089625    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-cvmg4"]
Oct 30 19:55:13.540753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:13.540676    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" path="/var/lib/kubelet/pods/098d5771-830f-45a6-acda-76ca9d7dd365/volumes"
Oct 30 19:55:15.350434 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.350398    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"]
Oct 30 19:55:15.351064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351017    5232 cpu_manager.go:414] "containerMap: removing container" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy"
Oct 30 19:55:15.351064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351039    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy"
Oct 30 19:55:15.351064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351057    5232 cpu_manager.go:414] "containerMap: removing container" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy"
Oct 30 19:55:15.351064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351066    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy"
Oct 30 19:55:15.351448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351081    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5ba1a1aa-85f3-4750-ba11-a3ac1b686779" containerName="maas-api"
Oct 30 19:55:15.351448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351090    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ba1a1aa-85f3-4750-ba11-a3ac1b686779" containerName="maas-api"
Oct 30 19:55:15.351448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351098    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1a60656a-6b08-4198-9649-23c3e07c4399" containerName="maas-api"
Oct 30 19:55:15.351448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351106    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a60656a-6b08-4198-9649-23c3e07c4399" containerName="maas-api"
Oct 30 19:55:15.351448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351317    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy"
Oct 30 19:55:15.351448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351331    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="098d5771-830f-45a6-acda-76ca9d7dd365" containerName="istio-proxy"
Oct 30 19:55:15.351448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351343    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ba1a1aa-85f3-4750-ba11-a3ac1b686779" containerName="maas-api"
Oct 30 19:55:15.351448 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.351351    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a60656a-6b08-4198-9649-23c3e07c4399" containerName="maas-api"
Oct 30 19:55:15.356757 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.356735    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.430531 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.430486    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.430531 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.430534    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bzdz\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-kube-api-access-7bzdz\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.430759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.430637    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.430759 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.430701    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.430889 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.430772    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.430889 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.430838    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.430889 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.430884    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c7681e-81cf-4bdb-b23b-004d3a614538-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.431012 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.430930    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.431062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.431050    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.458504 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.458458    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"]
Oct 30 19:55:15.532048 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532008    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532061    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532089    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bzdz\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-kube-api-access-7bzdz\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532126    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532183    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532215    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532269    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532304    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c7681e-81cf-4bdb-b23b-004d3a614538-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532533 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532351    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532931 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532902    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.532931 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.532914    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.533132 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.533066    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.533200 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.533151    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.533545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.533524    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c7681e-81cf-4bdb-b23b-004d3a614538-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.535399 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.535371    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.535522 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.535428    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.591393 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.591359    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.591552 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.591442    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bzdz\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-kube-api-access-7bzdz\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:15.670958 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:15.670871    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:16.102786 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:16.102747    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"]
Oct 30 19:55:16.106896 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:55:16.106857    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c7681e_81cf_4bdb_b23b_004d3a614538.slice/crio-55957763c0f7f63210d3691affedaacd8232e1534bd2e3460077d3f597f0e0fa WatchSource:0}: Error finding container 55957763c0f7f63210d3691affedaacd8232e1534bd2e3460077d3f597f0e0fa: Status 404 returned error can't find the container with id 55957763c0f7f63210d3691affedaacd8232e1534bd2e3460077d3f597f0e0fa
Oct 30 19:55:16.109823 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:16.109767    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:16.109941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:16.109876    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:16.109941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:16.109914    5232 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:16.191300 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:55:16.191269    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:55:16.944247 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:16.944201    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" event={"ID":"e9c7681e-81cf-4bdb-b23b-004d3a614538","Type":"ContainerStarted","Data":"02d6df983b880555fe25348293023321123e1265c765361b72c96da8f1721cb2"}
Oct 30 19:55:16.944748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:16.944264    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" event={"ID":"e9c7681e-81cf-4bdb-b23b-004d3a614538","Type":"ContainerStarted","Data":"55957763c0f7f63210d3691affedaacd8232e1534bd2e3460077d3f597f0e0fa"}
Oct 30 19:55:17.166710 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:17.166648    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" podStartSLOduration=2.166630219 podStartE2EDuration="2.166630219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:55:17.11360197 +0000 UTC m=+4109.698084968" watchObservedRunningTime="2026-10-30 19:55:17.166630219 +0000 UTC m=+4109.751113215"
Oct 30 19:55:17.671635 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:17.671597    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:17.673259 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:17.673229    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 19:55:17.673397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:17.673283    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 19:55:18.671229 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:18.671189    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 19:55:18.671646 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:18.671250    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 19:55:19.672067 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:19.672034    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 19:55:19.672486 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:19.672101    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 19:55:20.671365 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:20.671332    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 19:55:20.671549 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:20.671380    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 19:55:21.671940 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:21.671896    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 19:55:21.672361 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:21.671957    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 19:55:22.671619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:22.671580    5232 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused" start-of-body=
Oct 30 19:55:22.671893 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:22.671661    5232 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.178:15021/healthz/ready\": dial tcp 10.133.0.178:15021: connect: connection refused"
Oct 30 19:55:23.676643 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:23.676601    5232 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:23.679986 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:23.679953    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:23.684587 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:55:23.684556    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:55:46.188179 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:55:46.188135    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:56:06.199806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.199769    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"]
Oct 30 19:56:06.200334 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.200041    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8" podUID="4a517ee6-c6b5-4350-9823-1c3fddfd75c2" containerName="maas-api" containerID="cri-o://2aa6f18d8157e819a158534dd159876472204aac47e3b4e58021fc07aea83d69" gracePeriod=30
Oct 30 19:56:06.388174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.388143    5232 generic.go:356] "Generic (PLEG): container finished" podID="4a517ee6-c6b5-4350-9823-1c3fddfd75c2" containerID="2aa6f18d8157e819a158534dd159876472204aac47e3b4e58021fc07aea83d69" exitCode=0
Oct 30 19:56:06.388407 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.388378    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8" event={"ID":"4a517ee6-c6b5-4350-9823-1c3fddfd75c2","Type":"ContainerDied","Data":"2aa6f18d8157e819a158534dd159876472204aac47e3b4e58021fc07aea83d69"}
Oct 30 19:56:06.504773 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.504747    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:56:06.671102 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.671064    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-metrics-certs\") pod \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") "
Oct 30 19:56:06.671296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.671163    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-kube-api-access-wb2hm\" (UniqueName: \"kubernetes.io/projected/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-kube-api-access-wb2hm\") pod \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") "
Oct 30 19:56:06.671296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.671211    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-openshift-service-ca\") pod \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") "
Oct 30 19:56:06.671392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.671366    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls\") pod \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\" (UID: \"4a517ee6-c6b5-4350-9823-1c3fddfd75c2\") "
Oct 30 19:56:06.671714 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.671671    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-openshift-service-ca" pod "4a517ee6-c6b5-4350-9823-1c3fddfd75c2" (UID: "4a517ee6-c6b5-4350-9823-1c3fddfd75c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:06.672431 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.672411    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:06.673744 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.673714    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-kube-api-access-wb2hm" pod "4a517ee6-c6b5-4350-9823-1c3fddfd75c2" (UID: "4a517ee6-c6b5-4350-9823-1c3fddfd75c2"). InnerVolumeSpecName "kube-api-access-wb2hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:06.674060 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.674037    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls" pod "4a517ee6-c6b5-4350-9823-1c3fddfd75c2" (UID: "4a517ee6-c6b5-4350-9823-1c3fddfd75c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:06.674175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.674156    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-metrics-certs" pod "4a517ee6-c6b5-4350-9823-1c3fddfd75c2" (UID: "4a517ee6-c6b5-4350-9823-1c3fddfd75c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:06.773546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.773444    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:06.773546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.773476    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:06.773546 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:06.773488    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb2hm\" (UniqueName: \"kubernetes.io/projected/4a517ee6-c6b5-4350-9823-1c3fddfd75c2-kube-api-access-wb2hm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:07.396292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:07.396255    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"
Oct 30 19:56:07.396292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:07.396272    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8" event={"ID":"4a517ee6-c6b5-4350-9823-1c3fddfd75c2","Type":"ContainerDied","Data":"114aeaab533205c820b0b38f356e4024f33d6c0dd972d3dd027f7cb877eeb940"}
Oct 30 19:56:07.396872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:07.396333    5232 scope.go:122] "RemoveContainer" containerID="2aa6f18d8157e819a158534dd159876472204aac47e3b4e58021fc07aea83d69"
Oct 30 19:56:07.557004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:07.556963    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"]
Oct 30 19:56:07.604561 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:07.604521    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-905752-f4d85878c-2sbl8"]
Oct 30 19:56:09.541021 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:09.540983    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a517ee6-c6b5-4350-9823-1c3fddfd75c2" path="/var/lib/kubelet/pods/4a517ee6-c6b5-4350-9823-1c3fddfd75c2/volumes"
Oct 30 19:56:15.894543 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:15.894506    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"]
Oct 30 19:56:15.895011 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:15.894741    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy" containerID="cri-o://c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6" gracePeriod=30
Oct 30 19:56:16.187108 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:56:16.187019    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:56:21.171399 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.171330    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:56:21.265719 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.265682    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istiod-ca-cert\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.265923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.265733    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-kube-api-access-2xgbp\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-kube-api-access-2xgbp\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.265923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.265753    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-token\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-token\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.265923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.265845    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-socket\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.265923 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.265875    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-credential-socket\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.266158 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.265931    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-data\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.266158 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.265968    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-envoy\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.266158 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266094    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-certs\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.266158 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266122    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-podinfo\") pod \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\" (UID: \"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4\") "
Oct 30 19:56:21.266385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266180    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-socket" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:21.266385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266208    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-data" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:21.266385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266216    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istiod-ca-cert" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:21.266560 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266386    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-certs" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:21.266619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266571    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-credential-socket" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:21.266753 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266734    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.266853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266759    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.266853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266774    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.266853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266788    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.266853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.266803    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.268295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.268259    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-envoy" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:21.268397 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.268272    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-podinfo" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:56:21.268520 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.268502    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-token" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:21.268560 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.268515    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-kube-api-access-2xgbp" pod "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" (UID: "14d8f134-a0b0-4bb3-8f12-361e41b1e5d4"). InnerVolumeSpecName "kube-api-access-2xgbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:21.368270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.368220    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.368270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.368263    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.368270 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.368273    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xgbp\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-kube-api-access-2xgbp\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.368517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.368284    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:21.511660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.511557    5232 generic.go:356] "Generic (PLEG): container finished" podID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerID="c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6" exitCode=0
Oct 30 19:56:21.511660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.511615    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"
Oct 30 19:56:21.511660 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.511640    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" event={"ID":"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4","Type":"ContainerDied","Data":"c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6"}
Oct 30 19:56:21.511992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.511739    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9" event={"ID":"14d8f134-a0b0-4bb3-8f12-361e41b1e5d4","Type":"ContainerDied","Data":"43e5bbacecc41ff619f86d33c9dcc352e31028a36692315d765983cf38c8f235"}
Oct 30 19:56:21.511992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.511772    5232 scope.go:122] "RemoveContainer" containerID="c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6"
Oct 30 19:56:21.524034 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.524014    5232 scope.go:122] "RemoveContainer" containerID="07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2"
Oct 30 19:56:21.533900 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.533863    5232 scope.go:122] "RemoveContainer" containerID="c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6"
Oct 30 19:56:21.534227 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:56:21.534204    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6\": container with ID starting with c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6 not found: ID does not exist" containerID="c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6"
Oct 30 19:56:21.534282 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.534237    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6"} err="failed to get container status \"c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6\": rpc error: code = NotFound desc = could not find container \"c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6\": container with ID starting with c7b40135111d13155ec7f1ae7e27af8731af62cba8bbd15776ee94c5aa26e0c6 not found: ID does not exist"
Oct 30 19:56:21.534282 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.534253    5232 scope.go:122] "RemoveContainer" containerID="07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2"
Oct 30 19:56:21.534553 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:56:21.534533    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2\": container with ID starting with 07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2 not found: ID does not exist" containerID="07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2"
Oct 30 19:56:21.534617 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.534562    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2"} err="failed to get container status \"07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2\": rpc error: code = NotFound desc = could not find container \"07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2\": container with ID starting with 07bf6ec8ed4b8054dcabac8cd64a726459a3874c01d3d1b6d4d540b4fc68d7e2 not found: ID does not exist"
Oct 30 19:56:21.672077 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.672033    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"]
Oct 30 19:56:21.675600 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:21.675574    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-905752-openshift-default-56c768f8cc-q9np9"]
Oct 30 19:56:23.540466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:23.540426    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" path="/var/lib/kubelet/pods/14d8f134-a0b0-4bb3-8f12-361e41b1e5d4/volumes"
Oct 30 19:56:37.298212 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.298176    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"]
Oct 30 19:56:37.298994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.298973    5232 cpu_manager.go:414] "containerMap: removing container" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy"
Oct 30 19:56:37.299124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.298995    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy"
Oct 30 19:56:37.299124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.299022    5232 cpu_manager.go:414] "containerMap: removing container" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy"
Oct 30 19:56:37.299124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.299030    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy"
Oct 30 19:56:37.299124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.299043    5232 cpu_manager.go:414] "containerMap: removing container" podUID="4a517ee6-c6b5-4350-9823-1c3fddfd75c2" containerName="maas-api"
Oct 30 19:56:37.299124 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.299050    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a517ee6-c6b5-4350-9823-1c3fddfd75c2" containerName="maas-api"
Oct 30 19:56:37.299385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.299191    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy"
Oct 30 19:56:37.299385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.299207    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="14d8f134-a0b0-4bb3-8f12-361e41b1e5d4" containerName="istio-proxy"
Oct 30 19:56:37.299385 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.299217    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a517ee6-c6b5-4350-9823-1c3fddfd75c2" containerName="maas-api"
Oct 30 19:56:37.301927 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.301906    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.352653 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.352608    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-d50b07-serving-cert\""
Oct 30 19:56:37.378231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.378193    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-metrics-certs\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.378231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.378233    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.378498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.378306    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3422719-5018-4802-9df7-13a780733fbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.378498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.378339    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prg4x\" (UniqueName: \"kubernetes.io/projected/d3422719-5018-4802-9df7-13a780733fbb-kube-api-access-prg4x\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.403530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.403496    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"]
Oct 30 19:56:37.480164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.480131    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.480164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.480191    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3422719-5018-4802-9df7-13a780733fbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.480436 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.480236    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prg4x\" (UniqueName: \"kubernetes.io/projected/d3422719-5018-4802-9df7-13a780733fbb-kube-api-access-prg4x\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.480436 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:56:37.480256    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-serving-cert: secret "maas-api-e2e-extmod-a-d50b07-serving-cert" not found
Oct 30 19:56:37.480436 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:56:37.480320    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls podName:d3422719-5018-4802-9df7-13a780733fbb nodeName:}" failed. No retries permitted until 2026-10-30 19:56:37.980298987 +0000 UTC m=+4190.564781981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls") pod "maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57" (UID: "d3422719-5018-4802-9df7-13a780733fbb") : secret "maas-api-e2e-extmod-a-d50b07-serving-cert" not found
Oct 30 19:56:37.480649 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.480543    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-metrics-certs\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.480994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.480968    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3422719-5018-4802-9df7-13a780733fbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.483112 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.483092    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-metrics-certs\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.549593 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.549498    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prg4x\" (UniqueName: \"kubernetes.io/projected/d3422719-5018-4802-9df7-13a780733fbb-kube-api-access-prg4x\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.986883 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.986846    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:37.989343 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:37.989319    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls\") pod \"maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:38.214583 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:38.214549    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:38.590370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:38.590335    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"]
Oct 30 19:56:38.595233 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:56:38.595205    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3422719_5018_4802_9df7_13a780733fbb.slice/crio-28ea26a4ccd51743180add149ae222631fa086e62abf924565cb3827d29b92fe WatchSource:0}: Error finding container 28ea26a4ccd51743180add149ae222631fa086e62abf924565cb3827d29b92fe: Status 404 returned error can't find the container with id 28ea26a4ccd51743180add149ae222631fa086e62abf924565cb3827d29b92fe
Oct 30 19:56:38.641352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:38.641315    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57" event={"ID":"d3422719-5018-4802-9df7-13a780733fbb","Type":"ContainerStarted","Data":"28ea26a4ccd51743180add149ae222631fa086e62abf924565cb3827d29b92fe"}
Oct 30 19:56:39.649792 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:39.649694    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57" event={"ID":"d3422719-5018-4802-9df7-13a780733fbb","Type":"ContainerStarted","Data":"78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b"}
Oct 30 19:56:39.649792 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:39.649770    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:39.764536 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:39.764476    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57" podStartSLOduration=1.968341554 podStartE2EDuration="2.76446228s" totalImagesPullingTime="796.120726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:56:39.762882691 +0000 UTC m=+4192.347365689" watchObservedRunningTime="2026-10-30 19:56:39.76446228 +0000 UTC m=+4192.348945277"
Oct 30 19:56:45.134357 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:45.134322    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"]
Oct 30 19:56:45.134806 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:45.134529    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy" containerID="cri-o://6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9" gracePeriod=30
Oct 30 19:56:45.663936 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:45.663904    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 19:56:46.187852 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:56:46.187802    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:56:46.539267 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.539180    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"]
Oct 30 19:56:46.539662 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.539630    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh" podUID="519960a2-3a77-4228-8f22-0455fcd3cfa9" containerName="maas-api" containerID="cri-o://0a9f4a21debab5b00195580d809d8c117ffeeaf4ebf11eaa08f1101131f18669" gracePeriod=30
Oct 30 19:56:46.698094 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.698066    5232 generic.go:356] "Generic (PLEG): container finished" podID="519960a2-3a77-4228-8f22-0455fcd3cfa9" containerID="0a9f4a21debab5b00195580d809d8c117ffeeaf4ebf11eaa08f1101131f18669" exitCode=0
Oct 30 19:56:46.698271 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.698140    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh" event={"ID":"519960a2-3a77-4228-8f22-0455fcd3cfa9","Type":"ContainerDied","Data":"0a9f4a21debab5b00195580d809d8c117ffeeaf4ebf11eaa08f1101131f18669"}
Oct 30 19:56:46.819969 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.819932    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:56:46.898084 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.898046    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls\") pod \"519960a2-3a77-4228-8f22-0455fcd3cfa9\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") "
Oct 30 19:56:46.898264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.898105    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/519960a2-3a77-4228-8f22-0455fcd3cfa9-kube-api-access-kq8j9\" (UniqueName: \"kubernetes.io/projected/519960a2-3a77-4228-8f22-0455fcd3cfa9-kube-api-access-kq8j9\") pod \"519960a2-3a77-4228-8f22-0455fcd3cfa9\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") "
Oct 30 19:56:46.898264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.898124    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/519960a2-3a77-4228-8f22-0455fcd3cfa9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/519960a2-3a77-4228-8f22-0455fcd3cfa9-openshift-service-ca\") pod \"519960a2-3a77-4228-8f22-0455fcd3cfa9\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") "
Oct 30 19:56:46.898264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.898240    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-metrics-certs\") pod \"519960a2-3a77-4228-8f22-0455fcd3cfa9\" (UID: \"519960a2-3a77-4228-8f22-0455fcd3cfa9\") "
Oct 30 19:56:46.898615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.898590    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/519960a2-3a77-4228-8f22-0455fcd3cfa9-openshift-service-ca" pod "519960a2-3a77-4228-8f22-0455fcd3cfa9" (UID: "519960a2-3a77-4228-8f22-0455fcd3cfa9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:46.900281 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.900255    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-metrics-certs" pod "519960a2-3a77-4228-8f22-0455fcd3cfa9" (UID: "519960a2-3a77-4228-8f22-0455fcd3cfa9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:46.900392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.900303    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls" pod "519960a2-3a77-4228-8f22-0455fcd3cfa9" (UID: "519960a2-3a77-4228-8f22-0455fcd3cfa9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:46.900392 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:46.900308    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519960a2-3a77-4228-8f22-0455fcd3cfa9-kube-api-access-kq8j9" pod "519960a2-3a77-4228-8f22-0455fcd3cfa9" (UID: "519960a2-3a77-4228-8f22-0455fcd3cfa9"). InnerVolumeSpecName "kube-api-access-kq8j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:47.000311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.000274    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:47.000311 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.000308    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/519960a2-3a77-4228-8f22-0455fcd3cfa9-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:47.000527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.000323    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq8j9\" (UniqueName: \"kubernetes.io/projected/519960a2-3a77-4228-8f22-0455fcd3cfa9-kube-api-access-kq8j9\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:47.000527 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.000338    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/519960a2-3a77-4228-8f22-0455fcd3cfa9-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:47.708180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.708139    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh" event={"ID":"519960a2-3a77-4228-8f22-0455fcd3cfa9","Type":"ContainerDied","Data":"4c68daf882c83035618101569172655c7354c4cb3a2d67ed4c4b4d915f6b8609"}
Oct 30 19:56:47.708180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.708166    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"
Oct 30 19:56:47.708715 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.708217    5232 scope.go:122] "RemoveContainer" containerID="0a9f4a21debab5b00195580d809d8c117ffeeaf4ebf11eaa08f1101131f18669"
Oct 30 19:56:47.871674 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.871641    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"]
Oct 30 19:56:47.879728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:47.879690    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-660b59-6d968f89b9-sgbrh"]
Oct 30 19:56:48.200405 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:48.200322    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:56:48.211897 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:48.211856    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:56:48.241428 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:48.240121    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 19:56:48.246138 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:48.246106    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 19:56:49.539708 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:49.539629    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="519960a2-3a77-4228-8f22-0455fcd3cfa9" path="/var/lib/kubelet/pods/519960a2-3a77-4228-8f22-0455fcd3cfa9/volumes"
Oct 30 19:56:50.411748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.411673    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:56:50.558723 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.558684    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-data\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.558793    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-certs\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.558855    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-podinfo\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.558919    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-token\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-token\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.558953    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-envoy\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559026    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/494821ca-b40d-4423-b192-0ce5a101eeb4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/494821ca-b40d-4423-b192-0ce5a101eeb4-istiod-ca-cert\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559056    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-credential-socket\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559017    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-data" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559109    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-certs" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:50.559180 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559166    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-kube-api-access-tn5mb\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-kube-api-access-tn5mb\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559190    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-socket\") pod \"494821ca-b40d-4423-b192-0ce5a101eeb4\" (UID: \"494821ca-b40d-4423-b192-0ce5a101eeb4\") "
Oct 30 19:56:50.559784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559371    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-credential-socket" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:50.559784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559456    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/494821ca-b40d-4423-b192-0ce5a101eeb4-istiod-ca-cert" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:50.559784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559516    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-socket" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:50.559784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559775    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/494821ca-b40d-4423-b192-0ce5a101eeb4-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.560096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559798    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.560096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559834    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.560096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559851    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.560096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.559871    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.561242 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.561217    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-podinfo" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:56:50.561556 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.561534    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-token" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:50.561623 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.561565    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-kube-api-access-tn5mb" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "kube-api-access-tn5mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:50.561771 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.561752    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-envoy" pod "494821ca-b40d-4423-b192-0ce5a101eeb4" (UID: "494821ca-b40d-4423-b192-0ce5a101eeb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:50.661429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.661395    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn5mb\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-kube-api-access-tn5mb\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.661429 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.661428    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.661645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.661442    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.661645 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.661451    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/494821ca-b40d-4423-b192-0ce5a101eeb4-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:50.729735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.729648    5232 generic.go:356] "Generic (PLEG): container finished" podID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerID="6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9" exitCode=0
Oct 30 19:56:50.729735 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.729702    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"
Oct 30 19:56:50.729983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.729730    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" event={"ID":"494821ca-b40d-4423-b192-0ce5a101eeb4","Type":"ContainerDied","Data":"6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9"}
Oct 30 19:56:50.729983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.729784    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr" event={"ID":"494821ca-b40d-4423-b192-0ce5a101eeb4","Type":"ContainerDied","Data":"1b8be4a488c0a6aca6c17df0ffd33e72de9d5ff0340c760b40cdeef9724b642b"}
Oct 30 19:56:50.729983 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.729822    5232 scope.go:122] "RemoveContainer" containerID="6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9"
Oct 30 19:56:50.746779 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.746751    5232 scope.go:122] "RemoveContainer" containerID="c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6"
Oct 30 19:56:50.756551 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.756529    5232 scope.go:122] "RemoveContainer" containerID="6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9"
Oct 30 19:56:50.756865 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:56:50.756806    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9\": container with ID starting with 6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9 not found: ID does not exist" containerID="6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9"
Oct 30 19:56:50.756931 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.756877    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9"} err="failed to get container status \"6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9\": rpc error: code = NotFound desc = could not find container \"6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9\": container with ID starting with 6e0e8fffbf08f1268b169b3fb0f1b5f0063c970b8c7d15cd068f5ae214ce87d9 not found: ID does not exist"
Oct 30 19:56:50.756931 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.756901    5232 scope.go:122] "RemoveContainer" containerID="c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6"
Oct 30 19:56:50.757153 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:56:50.757132    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6\": container with ID starting with c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6 not found: ID does not exist" containerID="c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6"
Oct 30 19:56:50.757212 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.757163    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6"} err="failed to get container status \"c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6\": rpc error: code = NotFound desc = could not find container \"c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6\": container with ID starting with c6dd4a282ecad779b9841457d1c4577a3e43d3fbfe32e0b9bb9fdc1b54c1bee6 not found: ID does not exist"
Oct 30 19:56:50.891402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.891362    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"]
Oct 30 19:56:50.894798 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:50.894770    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a36b7-openshift-default-777d7b7f89-7r6lr"]
Oct 30 19:56:51.540865 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:51.540834    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" path="/var/lib/kubelet/pods/494821ca-b40d-4423-b192-0ce5a101eeb4/volumes"
Oct 30 19:56:57.677687 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:57.677653    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"]
Oct 30 19:56:57.678252 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:57.677939    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy" containerID="cri-o://02d6df983b880555fe25348293023321123e1265c765361b72c96da8f1721cb2" gracePeriod=30
Oct 30 19:56:57.701169 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:57.701144    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"]
Oct 30 19:56:57.701340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:56:57.701321    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy" containerID="cri-o://bfce85e8cda32be67cc186faf3b4a0f22d70d7aba0e7d1cc93a04a842cc5457f" gracePeriod=30
Oct 30 19:57:02.847370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:02.847329    5232 generic.go:356] "Generic (PLEG): container finished" podID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerID="02d6df983b880555fe25348293023321123e1265c765361b72c96da8f1721cb2" exitCode=0
Oct 30 19:57:02.848003 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:02.847506    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" event={"ID":"e9c7681e-81cf-4bdb-b23b-004d3a614538","Type":"ContainerDied","Data":"02d6df983b880555fe25348293023321123e1265c765361b72c96da8f1721cb2"}
Oct 30 19:57:02.850461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:02.850410    5232 generic.go:356] "Generic (PLEG): container finished" podID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerID="bfce85e8cda32be67cc186faf3b4a0f22d70d7aba0e7d1cc93a04a842cc5457f" exitCode=0
Oct 30 19:57:02.850636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:02.850484    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" event={"ID":"5401cf32-a2fd-4804-b703-5cf2116c6510","Type":"ContainerDied","Data":"bfce85e8cda32be67cc186faf3b4a0f22d70d7aba0e7d1cc93a04a842cc5457f"}
Oct 30 19:57:03.010272 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.009887    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:57:03.031736 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.031704    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:57:03.112740 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.112701    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-envoy\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.112962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.112777    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-envoy\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.112962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.112864    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-podinfo\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.112962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.112901    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-socket\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.112962 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.112935    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-credential-socket\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113183    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-socket" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113221    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-credential-socket" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113276    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-socket\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113351    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-token\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-token\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113398    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-certs\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113460    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-data\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113513    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-certs\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113560    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-token\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-token\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113587    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9c7681e-81cf-4bdb-b23b-004d3a614538-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c7681e-81cf-4bdb-b23b-004d3a614538-istiod-ca-cert\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.113691 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113663    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-kube-api-access-x5jk8\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-kube-api-access-x5jk8\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.114367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113717    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-kube-api-access-7bzdz\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-kube-api-access-7bzdz\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.114367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113748    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-podinfo\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.114367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113788    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-data\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.114367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113840    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-credential-socket\") pod \"e9c7681e-81cf-4bdb-b23b-004d3a614538\" (UID: \"e9c7681e-81cf-4bdb-b23b-004d3a614538\") "
Oct 30 19:57:03.114367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113871    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5401cf32-a2fd-4804-b703-5cf2116c6510-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5401cf32-a2fd-4804-b703-5cf2116c6510-istiod-ca-cert\") pod \"5401cf32-a2fd-4804-b703-5cf2116c6510\" (UID: \"5401cf32-a2fd-4804-b703-5cf2116c6510\") "
Oct 30 19:57:03.114367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.113876    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-certs" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.114367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114077    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-certs" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.114367 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114290    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-data" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.114807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114450    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5401cf32-a2fd-4804-b703-5cf2116c6510-istiod-ca-cert" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:03.114807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114724    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.114807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114735    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-data" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.114807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114745    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.114807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114779    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.114807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114799    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5401cf32-a2fd-4804-b703-5cf2116c6510-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.114807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114850    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.114807 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114865    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.115517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.114950    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-socket" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.115517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.115021    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-credential-socket" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.115517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.115258    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c7681e-81cf-4bdb-b23b-004d3a614538-istiod-ca-cert" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:03.115517 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.115458    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-envoy" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.116236 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.116203    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-podinfo" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:57:03.117062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.116986    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-token" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:03.117174 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.117069    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-envoy" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.117742 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.117714    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-kube-api-access-7bzdz" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "kube-api-access-7bzdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:03.118078 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.118059    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-token" pod "e9c7681e-81cf-4bdb-b23b-004d3a614538" (UID: "e9c7681e-81cf-4bdb-b23b-004d3a614538"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:03.118305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.118278    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-kube-api-access-x5jk8" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "kube-api-access-x5jk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:03.118598 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.118581    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-podinfo" pod "5401cf32-a2fd-4804-b703-5cf2116c6510" (UID: "5401cf32-a2fd-4804-b703-5cf2116c6510"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:57:03.215748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215707    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.215748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215740    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c7681e-81cf-4bdb-b23b-004d3a614538-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.215748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215751    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5jk8\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-kube-api-access-x5jk8\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215761    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bzdz\" (UniqueName: \"kubernetes.io/projected/e9c7681e-81cf-4bdb-b23b-004d3a614538-kube-api-access-7bzdz\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215771    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215779    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215788    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215796    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215806    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215826    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c7681e-81cf-4bdb-b23b-004d3a614538-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215834    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c7681e-81cf-4bdb-b23b-004d3a614538-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.216103 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.215842    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5401cf32-a2fd-4804-b703-5cf2116c6510-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.858144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.858113    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"
Oct 30 19:57:03.858667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.858136    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb" event={"ID":"e9c7681e-81cf-4bdb-b23b-004d3a614538","Type":"ContainerDied","Data":"55957763c0f7f63210d3691affedaacd8232e1534bd2e3460077d3f597f0e0fa"}
Oct 30 19:57:03.858667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.858200    5232 scope.go:122] "RemoveContainer" containerID="02d6df983b880555fe25348293023321123e1265c765361b72c96da8f1721cb2"
Oct 30 19:57:03.859703 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.859682    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"
Oct 30 19:57:03.859831 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.859715    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk" event={"ID":"5401cf32-a2fd-4804-b703-5cf2116c6510","Type":"ContainerDied","Data":"6680a1a65f38dc2b78d7ff331b33f81bd278c49612f6d9065ef5e820ae7dca41"}
Oct 30 19:57:03.868550 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:03.868525    5232 scope.go:122] "RemoveContainer" containerID="bfce85e8cda32be67cc186faf3b4a0f22d70d7aba0e7d1cc93a04a842cc5457f"
Oct 30 19:57:04.024795 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:04.024497    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"]
Oct 30 19:57:04.071798 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:04.071763    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-trxzk"]
Oct 30 19:57:04.177173 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:04.177099    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"]
Oct 30 19:57:04.186279 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:04.186230    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-blglb"]
Oct 30 19:57:05.540498 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:05.540465    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" path="/var/lib/kubelet/pods/5401cf32-a2fd-4804-b703-5cf2116c6510/volumes"
Oct 30 19:57:05.540993 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:05.540978    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" path="/var/lib/kubelet/pods/e9c7681e-81cf-4bdb-b23b-004d3a614538/volumes"
Oct 30 19:57:16.187210 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:57:16.187174    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:57:17.870919 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:17.870887    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"]
Oct 30 19:57:17.871376 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:17.871125    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr" podUID="a51573e1-e9cb-44ef-94d7-c25d887678a4" containerName="maas-api" containerID="cri-o://d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029" gracePeriod=30
Oct 30 19:57:18.133632 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.133573    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:57:18.211167 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.211137    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls\") pod \"a51573e1-e9cb-44ef-94d7-c25d887678a4\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") "
Oct 30 19:57:18.211331 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.211255    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-metrics-certs\") pod \"a51573e1-e9cb-44ef-94d7-c25d887678a4\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") "
Oct 30 19:57:18.211331 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.211304    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a51573e1-e9cb-44ef-94d7-c25d887678a4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a51573e1-e9cb-44ef-94d7-c25d887678a4-openshift-service-ca\") pod \"a51573e1-e9cb-44ef-94d7-c25d887678a4\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") "
Oct 30 19:57:18.211415 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.211339    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a51573e1-e9cb-44ef-94d7-c25d887678a4-kube-api-access-wzt59\" (UniqueName: \"kubernetes.io/projected/a51573e1-e9cb-44ef-94d7-c25d887678a4-kube-api-access-wzt59\") pod \"a51573e1-e9cb-44ef-94d7-c25d887678a4\" (UID: \"a51573e1-e9cb-44ef-94d7-c25d887678a4\") "
Oct 30 19:57:18.211622 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.211588    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a51573e1-e9cb-44ef-94d7-c25d887678a4-openshift-service-ca" pod "a51573e1-e9cb-44ef-94d7-c25d887678a4" (UID: "a51573e1-e9cb-44ef-94d7-c25d887678a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:18.211862 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.211808    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a51573e1-e9cb-44ef-94d7-c25d887678a4-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:18.213407 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.213382    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls" pod "a51573e1-e9cb-44ef-94d7-c25d887678a4" (UID: "a51573e1-e9cb-44ef-94d7-c25d887678a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:57:18.213496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.213404    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-metrics-certs" pod "a51573e1-e9cb-44ef-94d7-c25d887678a4" (UID: "a51573e1-e9cb-44ef-94d7-c25d887678a4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:57:18.213496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.213422    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51573e1-e9cb-44ef-94d7-c25d887678a4-kube-api-access-wzt59" pod "a51573e1-e9cb-44ef-94d7-c25d887678a4" (UID: "a51573e1-e9cb-44ef-94d7-c25d887678a4"). InnerVolumeSpecName "kube-api-access-wzt59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:18.313449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.313414    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:18.313449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.313443    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a51573e1-e9cb-44ef-94d7-c25d887678a4-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:18.313449 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.313453    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzt59\" (UniqueName: \"kubernetes.io/projected/a51573e1-e9cb-44ef-94d7-c25d887678a4-kube-api-access-wzt59\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:18.959805 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.959770    5232 generic.go:356] "Generic (PLEG): container finished" podID="a51573e1-e9cb-44ef-94d7-c25d887678a4" containerID="d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029" exitCode=0
Oct 30 19:57:18.960294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.959830    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"
Oct 30 19:57:18.960294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.959841    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr" event={"ID":"a51573e1-e9cb-44ef-94d7-c25d887678a4","Type":"ContainerDied","Data":"d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029"}
Oct 30 19:57:18.960294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.959905    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr" event={"ID":"a51573e1-e9cb-44ef-94d7-c25d887678a4","Type":"ContainerDied","Data":"480295a280ab295b733e1e726f730ea72f673e81c2f3dbf256522a5b8bc47822"}
Oct 30 19:57:18.960294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.959936    5232 scope.go:122] "RemoveContainer" containerID="d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029"
Oct 30 19:57:18.970942 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.970924    5232 scope.go:122] "RemoveContainer" containerID="d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029"
Oct 30 19:57:18.971210 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:57:18.971189    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029\": container with ID starting with d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029 not found: ID does not exist" containerID="d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029"
Oct 30 19:57:18.971254 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:18.971219    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029"} err="failed to get container status \"d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029\": rpc error: code = NotFound desc = could not find container \"d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029\": container with ID starting with d64377f6ff6f2085c0cdf18fac0b2d0ecb010bf34d8618db3eb951dafd468029 not found: ID does not exist"
Oct 30 19:57:19.119567 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:19.119518    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"]
Oct 30 19:57:19.123711 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:19.123686    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2186d8-f7d5c7444-k8snr"]
Oct 30 19:57:19.539483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:19.539449    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a51573e1-e9cb-44ef-94d7-c25d887678a4" path="/var/lib/kubelet/pods/a51573e1-e9cb-44ef-94d7-c25d887678a4/volumes"
Oct 30 19:57:27.801888 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:27.801796    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"]
Oct 30 19:57:27.802328 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:27.802014    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy" containerID="cri-o://f0c51cf12ed86f68af9443a861e7d5ee8d9d5e4a151bcd4d6a9a4e087950710d" gracePeriod=30
Oct 30 19:57:33.053370 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.053336    5232 generic.go:356] "Generic (PLEG): container finished" podID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerID="f0c51cf12ed86f68af9443a861e7d5ee8d9d5e4a151bcd4d6a9a4e087950710d" exitCode=0
Oct 30 19:57:33.053868 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.053384    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" event={"ID":"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38","Type":"ContainerDied","Data":"f0c51cf12ed86f68af9443a861e7d5ee8d9d5e4a151bcd4d6a9a4e087950710d"}
Oct 30 19:57:33.053868 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.053450    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj" event={"ID":"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38","Type":"ContainerDied","Data":"200d1e7bed50ca79f912fd2ccdcf73ad467a04759e344be5a8bdf19fa3fe9f2b"}
Oct 30 19:57:33.053868 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.053485    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="200d1e7bed50ca79f912fd2ccdcf73ad467a04759e344be5a8bdf19fa3fe9f2b"
Oct 30 19:57:33.053868 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.053499    5232 scope.go:122] "RemoveContainer" containerID="6224ebe6a7c20904a7e85aa0e355c9636de85ea3afc8a60498135accb5453b65"
Oct 30 19:57:33.072394 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.072373    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:57:33.190481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190443    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-data\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.190683 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190557    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-certs\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.190683 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190594    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istiod-ca-cert\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.190683 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190621    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-credential-socket\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.190909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190699    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-token\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-token\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.190909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190751    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-socket\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.190909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190757    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-data" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:33.190909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190803    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-kube-api-access-46p9v\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-kube-api-access-46p9v\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.190909 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190851    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-certs" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:33.191164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190914    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-podinfo\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.191164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.190973    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-envoy\") pod \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\" (UID: \"78cb9d9c-5179-4dd3-a31f-a71b9bff4f38\") "
Oct 30 19:57:33.191164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.191016    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-credential-socket" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:33.191164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.191049    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-socket" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:33.191164 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.191093    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istiod-ca-cert" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:33.191438 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.191419    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.191503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.191443    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.191503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.191458    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.191503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.191474    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.191503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.191487    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.193112 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.193087    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-podinfo" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:57:33.193234 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.193120    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-kube-api-access-46p9v" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "kube-api-access-46p9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:33.193398 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.193375    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-envoy" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:33.193507 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.193405    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-token" pod "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" (UID: "78cb9d9c-5179-4dd3-a31f-a71b9bff4f38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:33.292572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.292536    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.292572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.292570    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46p9v\" (UniqueName: \"kubernetes.io/projected/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-kube-api-access-46p9v\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.292572 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.292581    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.292824 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.292590    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:33.871659 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.871615    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"]
Oct 30 19:57:33.871914 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.871863    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc" podUID="525f236d-76e8-4404-9c8c-5af6416906de" containerName="maas-api" containerID="cri-o://939648a92fe113f2d37a189cbc12d58d78fec1cefc3f81d29ba81d6b5b7d6346" gracePeriod=30
Oct 30 19:57:33.891851 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:33.891789    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc" podUID="525f236d-76e8-4404-9c8c-5af6416906de" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.150:8443/health\": dial tcp 10.133.0.150:8443: connect: connection refused"
Oct 30 19:57:34.066855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.066434    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"
Oct 30 19:57:34.068731 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.068682    5232 generic.go:356] "Generic (PLEG): container finished" podID="525f236d-76e8-4404-9c8c-5af6416906de" containerID="939648a92fe113f2d37a189cbc12d58d78fec1cefc3f81d29ba81d6b5b7d6346" exitCode=0
Oct 30 19:57:34.068915 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.068742    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc" event={"ID":"525f236d-76e8-4404-9c8c-5af6416906de","Type":"ContainerDied","Data":"939648a92fe113f2d37a189cbc12d58d78fec1cefc3f81d29ba81d6b5b7d6346"}
Oct 30 19:57:34.140541 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.140510    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:57:34.303347 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.303313    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"]
Oct 30 19:57:34.304363 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.304336    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-metrics-certs\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-metrics-certs\") pod \"525f236d-76e8-4404-9c8c-5af6416906de\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") "
Oct 30 19:57:34.304494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.304412    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/525f236d-76e8-4404-9c8c-5af6416906de-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/525f236d-76e8-4404-9c8c-5af6416906de-openshift-service-ca\") pod \"525f236d-76e8-4404-9c8c-5af6416906de\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") "
Oct 30 19:57:34.304494 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.304465    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls\") pod \"525f236d-76e8-4404-9c8c-5af6416906de\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") "
Oct 30 19:57:34.304669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.304652    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/525f236d-76e8-4404-9c8c-5af6416906de-kube-api-access-tn8nr\" (UniqueName: \"kubernetes.io/projected/525f236d-76e8-4404-9c8c-5af6416906de-kube-api-access-tn8nr\") pod \"525f236d-76e8-4404-9c8c-5af6416906de\" (UID: \"525f236d-76e8-4404-9c8c-5af6416906de\") "
Oct 30 19:57:34.305239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.304893    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/525f236d-76e8-4404-9c8c-5af6416906de-openshift-service-ca" pod "525f236d-76e8-4404-9c8c-5af6416906de" (UID: "525f236d-76e8-4404-9c8c-5af6416906de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:34.305239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.305209    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/525f236d-76e8-4404-9c8c-5af6416906de-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:34.306728 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.306702    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-metrics-certs" pod "525f236d-76e8-4404-9c8c-5af6416906de" (UID: "525f236d-76e8-4404-9c8c-5af6416906de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:57:34.306872 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.306769    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls" pod "525f236d-76e8-4404-9c8c-5af6416906de" (UID: "525f236d-76e8-4404-9c8c-5af6416906de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:57:34.307176 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.307158    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525f236d-76e8-4404-9c8c-5af6416906de-kube-api-access-tn8nr" pod "525f236d-76e8-4404-9c8c-5af6416906de" (UID: "525f236d-76e8-4404-9c8c-5af6416906de"). InnerVolumeSpecName "kube-api-access-tn8nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:34.308752 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.308728    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2186d8-openshift-default-5dc5c59749-x6mjj"]
Oct 30 19:57:34.406880 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.406785    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn8nr\" (UniqueName: \"kubernetes.io/projected/525f236d-76e8-4404-9c8c-5af6416906de-kube-api-access-tn8nr\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:34.406880 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.406860    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:34.406880 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:34.406878    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/525f236d-76e8-4404-9c8c-5af6416906de-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:35.077047 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:35.077013    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"
Oct 30 19:57:35.077523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:35.077054    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc" event={"ID":"525f236d-76e8-4404-9c8c-5af6416906de","Type":"ContainerDied","Data":"3d2721c5f3ad01a8ac7ffc57a39df1872ace2b3a7eb2c82fb3aa0317e70dc710"}
Oct 30 19:57:35.077523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:35.077130    5232 scope.go:122] "RemoveContainer" containerID="939648a92fe113f2d37a189cbc12d58d78fec1cefc3f81d29ba81d6b5b7d6346"
Oct 30 19:57:35.237227 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:35.237194    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"]
Oct 30 19:57:35.284425 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:35.284394    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-db06cf-677f8d78f-j4flc"]
Oct 30 19:57:35.545916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:35.545876    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="525f236d-76e8-4404-9c8c-5af6416906de" path="/var/lib/kubelet/pods/525f236d-76e8-4404-9c8c-5af6416906de/volumes"
Oct 30 19:57:35.546379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:35.546363    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" path="/var/lib/kubelet/pods/78cb9d9c-5179-4dd3-a31f-a71b9bff4f38/volumes"
Oct 30 19:57:46.187317 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:57:46.187275    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:57:49.206926 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:57:49.206896    5232 scope.go:122] "RemoveContainer" containerID="f0c51cf12ed86f68af9443a861e7d5ee8d9d5e4a151bcd4d6a9a4e087950710d"
Oct 30 19:58:03.549412 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.549372    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"]
Oct 30 19:58:03.549969 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.549643    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr" podUID="330a6418-9381-4c85-a56c-5e238567f9f0" containerName="maas-api" containerID="cri-o://5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803" gracePeriod=30
Oct 30 19:58:03.822381 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.822349    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:58:03.969789 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.969754    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/330a6418-9381-4c85-a56c-5e238567f9f0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/330a6418-9381-4c85-a56c-5e238567f9f0-openshift-service-ca\") pod \"330a6418-9381-4c85-a56c-5e238567f9f0\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") "
Oct 30 19:58:03.970023 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.969847    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls\") pod \"330a6418-9381-4c85-a56c-5e238567f9f0\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") "
Oct 30 19:58:03.970023 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.969894    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-metrics-certs\") pod \"330a6418-9381-4c85-a56c-5e238567f9f0\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") "
Oct 30 19:58:03.970023 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.969938    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/330a6418-9381-4c85-a56c-5e238567f9f0-kube-api-access-xn8cb\" (UniqueName: \"kubernetes.io/projected/330a6418-9381-4c85-a56c-5e238567f9f0-kube-api-access-xn8cb\") pod \"330a6418-9381-4c85-a56c-5e238567f9f0\" (UID: \"330a6418-9381-4c85-a56c-5e238567f9f0\") "
Oct 30 19:58:03.970218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.970179    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/330a6418-9381-4c85-a56c-5e238567f9f0-openshift-service-ca" pod "330a6418-9381-4c85-a56c-5e238567f9f0" (UID: "330a6418-9381-4c85-a56c-5e238567f9f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:03.970496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.970469    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/330a6418-9381-4c85-a56c-5e238567f9f0-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:03.972128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.972101    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls" pod "330a6418-9381-4c85-a56c-5e238567f9f0" (UID: "330a6418-9381-4c85-a56c-5e238567f9f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:03.972228 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.972199    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/330a6418-9381-4c85-a56c-5e238567f9f0-kube-api-access-xn8cb" pod "330a6418-9381-4c85-a56c-5e238567f9f0" (UID: "330a6418-9381-4c85-a56c-5e238567f9f0"). InnerVolumeSpecName "kube-api-access-xn8cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:03.972274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:03.972232    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-metrics-certs" pod "330a6418-9381-4c85-a56c-5e238567f9f0" (UID: "330a6418-9381-4c85-a56c-5e238567f9f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:04.072224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.072125    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:04.072224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.072161    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/330a6418-9381-4c85-a56c-5e238567f9f0-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:04.072224 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.072177    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn8cb\" (UniqueName: \"kubernetes.io/projected/330a6418-9381-4c85-a56c-5e238567f9f0-kube-api-access-xn8cb\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:04.269100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.269064    5232 generic.go:356] "Generic (PLEG): container finished" podID="330a6418-9381-4c85-a56c-5e238567f9f0" containerID="5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803" exitCode=0
Oct 30 19:58:04.269290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.269107    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"
Oct 30 19:58:04.269290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.269147    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr" event={"ID":"330a6418-9381-4c85-a56c-5e238567f9f0","Type":"ContainerDied","Data":"5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803"}
Oct 30 19:58:04.269290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.269206    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr" event={"ID":"330a6418-9381-4c85-a56c-5e238567f9f0","Type":"ContainerDied","Data":"45f724f2c41d401b87368216a03889effd2d0f5badae1faadec1dac772f7882e"}
Oct 30 19:58:04.269290 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.269234    5232 scope.go:122] "RemoveContainer" containerID="5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803"
Oct 30 19:58:04.282788 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.282761    5232 scope.go:122] "RemoveContainer" containerID="5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803"
Oct 30 19:58:04.283832 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:04.283791    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803\": container with ID starting with 5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803 not found: ID does not exist" containerID="5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803"
Oct 30 19:58:04.283898 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.283841    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803"} err="failed to get container status \"5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803\": rpc error: code = NotFound desc = could not find container \"5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803\": container with ID starting with 5a3ef05fd51cee415790877046a7e69f81c1e93f6c2bb296e43592e4e4a75803 not found: ID does not exist"
Oct 30 19:58:04.427590 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.427550    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"]
Oct 30 19:58:04.432151 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:04.432125    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-312161-6ddbfd954-hk8xr"]
Oct 30 19:58:05.539604 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:05.539566    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="330a6418-9381-4c85-a56c-5e238567f9f0" path="/var/lib/kubelet/pods/330a6418-9381-4c85-a56c-5e238567f9f0/volumes"
Oct 30 19:58:09.815353 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:09.815319    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"]
Oct 30 19:58:09.815799 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:09.815531    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd" podUID="9dbdae8e-cabf-4056-8f45-550d43dafa5f" containerName="maas-api" containerID="cri-o://8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2" gracePeriod=30
Oct 30 19:58:10.074156 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.074094    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:58:10.141074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.141036    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls\") pod \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") "
Oct 30 19:58:10.141262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.141115    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dbdae8e-cabf-4056-8f45-550d43dafa5f-kube-api-access-kv2gq\" (UniqueName: \"kubernetes.io/projected/9dbdae8e-cabf-4056-8f45-550d43dafa5f-kube-api-access-kv2gq\") pod \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") "
Oct 30 19:58:10.141262 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.141230    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dbdae8e-cabf-4056-8f45-550d43dafa5f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbdae8e-cabf-4056-8f45-550d43dafa5f-openshift-service-ca\") pod \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") "
Oct 30 19:58:10.141357 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.141270    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-metrics-certs\") pod \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\" (UID: \"9dbdae8e-cabf-4056-8f45-550d43dafa5f\") "
Oct 30 19:58:10.141614 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.141585    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dbdae8e-cabf-4056-8f45-550d43dafa5f-openshift-service-ca" pod "9dbdae8e-cabf-4056-8f45-550d43dafa5f" (UID: "9dbdae8e-cabf-4056-8f45-550d43dafa5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:10.143481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.143437    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dbdae8e-cabf-4056-8f45-550d43dafa5f-kube-api-access-kv2gq" pod "9dbdae8e-cabf-4056-8f45-550d43dafa5f" (UID: "9dbdae8e-cabf-4056-8f45-550d43dafa5f"). InnerVolumeSpecName "kube-api-access-kv2gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:10.143481 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.143465    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-metrics-certs" pod "9dbdae8e-cabf-4056-8f45-550d43dafa5f" (UID: "9dbdae8e-cabf-4056-8f45-550d43dafa5f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:10.143642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.143520    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls" pod "9dbdae8e-cabf-4056-8f45-550d43dafa5f" (UID: "9dbdae8e-cabf-4056-8f45-550d43dafa5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:10.242276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.242240    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv2gq\" (UniqueName: \"kubernetes.io/projected/9dbdae8e-cabf-4056-8f45-550d43dafa5f-kube-api-access-kv2gq\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:10.242276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.242269    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dbdae8e-cabf-4056-8f45-550d43dafa5f-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:10.242276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.242281    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:10.242519 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.242290    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dbdae8e-cabf-4056-8f45-550d43dafa5f-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:10.307784 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.307751    5232 generic.go:356] "Generic (PLEG): container finished" podID="9dbdae8e-cabf-4056-8f45-550d43dafa5f" containerID="8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2" exitCode=0
Oct 30 19:58:10.307996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.307796    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"
Oct 30 19:58:10.307996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.307842    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd" event={"ID":"9dbdae8e-cabf-4056-8f45-550d43dafa5f","Type":"ContainerDied","Data":"8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2"}
Oct 30 19:58:10.307996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.307908    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd" event={"ID":"9dbdae8e-cabf-4056-8f45-550d43dafa5f","Type":"ContainerDied","Data":"b90281845db676051ee79e2636648e764ca174d5ed2ef99f61c291566179595f"}
Oct 30 19:58:10.307996 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.307936    5232 scope.go:122] "RemoveContainer" containerID="8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2"
Oct 30 19:58:10.318001 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.317978    5232 scope.go:122] "RemoveContainer" containerID="8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2"
Oct 30 19:58:10.318271 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:10.318251    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2\": container with ID starting with 8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2 not found: ID does not exist" containerID="8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2"
Oct 30 19:58:10.318329 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.318280    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2"} err="failed to get container status \"8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2\": rpc error: code = NotFound desc = could not find container \"8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2\": container with ID starting with 8935b957ba37ed3cb5a154fa3b453e8b8fb78418666e6297dfe87fba3f82b9a2 not found: ID does not exist"
Oct 30 19:58:10.471051 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.471016    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"]
Oct 30 19:58:10.473766 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:10.473740    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-59594f-548665c65-gftbd"]
Oct 30 19:58:11.540422 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:11.540384    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dbdae8e-cabf-4056-8f45-550d43dafa5f" path="/var/lib/kubelet/pods/9dbdae8e-cabf-4056-8f45-550d43dafa5f/volumes"
Oct 30 19:58:14.877292 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:14.877259    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"]
Oct 30 19:58:14.877874 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:14.877506    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy" containerID="cri-o://617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314" gracePeriod=30
Oct 30 19:58:16.187729 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:16.187691    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:58:17.972027 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:17.971992    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"]
Oct 30 19:58:18.019992 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:18.019964    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"]
Oct 30 19:58:20.171331 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.171307    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:58:20.256461 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256427    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-socket\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.256663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256492    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-kube-api-access-js7c4\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-kube-api-access-js7c4\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.256663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256534    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-certs\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.256663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256568    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-data\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.256663 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256641    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-podinfo\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256707    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-envoy\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256735    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istiod-ca-cert\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256864    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-credential-socket\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256870    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-certs" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256885    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-socket" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256931    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-data" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.256942    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-token\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-token\") pod \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\" (UID: \"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb\") "
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.257211    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-credential-socket" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:20.257352 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.257217    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istiod-ca-cert" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:20.257758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.257529    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.257758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.257551    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.257758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.257566    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.257758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.257581    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.257758 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.257595    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.259099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.259072    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-token" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:20.259099 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.259075    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-envoy" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:20.259276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.259225    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-kube-api-access-js7c4" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "kube-api-access-js7c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:20.259276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.259228    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-podinfo" pod "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" (UID: "c50ac16c-f6e9-42f5-8e35-2cab4009e3bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:58:20.358827 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.358784    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js7c4\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-kube-api-access-js7c4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.359004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.358850    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.359004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.358867    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.359004 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.358884    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.372705 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.372666    5232 generic.go:356] "Generic (PLEG): container finished" podID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerID="617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314" exitCode=0
Oct 30 19:58:20.372934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.372731    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"
Oct 30 19:58:20.372934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.372749    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" event={"ID":"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb","Type":"ContainerDied","Data":"617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314"}
Oct 30 19:58:20.372934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.372838    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts" event={"ID":"c50ac16c-f6e9-42f5-8e35-2cab4009e3bb","Type":"ContainerDied","Data":"cb7a91794f16f9fb86b04d8d931d0023bc738d1ee405c7e7fb696a89769b8aaa"}
Oct 30 19:58:20.372934 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.372873    5232 scope.go:122] "RemoveContainer" containerID="617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314"
Oct 30 19:58:20.387100 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.387076    5232 scope.go:122] "RemoveContainer" containerID="cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f"
Oct 30 19:58:20.396899 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.396845    5232 scope.go:122] "RemoveContainer" containerID="617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314"
Oct 30 19:58:20.397930 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:20.397180    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314\": container with ID starting with 617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314 not found: ID does not exist" containerID="617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314"
Oct 30 19:58:20.397930 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.397216    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314"} err="failed to get container status \"617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314\": rpc error: code = NotFound desc = could not find container \"617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314\": container with ID starting with 617a54e7add4ca4dca39e3154bd8e900582be9e2a7edbfdf60d1b81d1fc56314 not found: ID does not exist"
Oct 30 19:58:20.397930 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.397241    5232 scope.go:122] "RemoveContainer" containerID="cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f"
Oct 30 19:58:20.397930 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:20.397567    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f\": container with ID starting with cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f not found: ID does not exist" containerID="cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f"
Oct 30 19:58:20.397930 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.397604    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f"} err="failed to get container status \"cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f\": rpc error: code = NotFound desc = could not find container \"cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f\": container with ID starting with cdbb4dd28c0111058aea428b32d75f34f1728a6fb0507d473ef0f4016e324b3f not found: ID does not exist"
Oct 30 19:58:20.404122 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.404097    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"]
Oct 30 19:58:20.405866 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.404308    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy" containerID="cri-o://683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46" gracePeriod=30
Oct 30 19:58:20.536263 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.536227    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"]
Oct 30 19:58:20.552314 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:20.552276    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-312161-openshift-default-68dc74f8b9-nxxts"]
Oct 30 19:58:21.540523 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:21.540486    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" path="/var/lib/kubelet/pods/c50ac16c-f6e9-42f5-8e35-2cab4009e3bb/volumes"
Oct 30 19:58:25.668953 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.668923    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:58:25.813416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.813305    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"]
Oct 30 19:58:25.815252 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815222    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-socket\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.815364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815317    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-credential-socket\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.815364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815353    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-token\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-token\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.815474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815393    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-data\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.815474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815434    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-certs\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.815474 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815461    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-kube-api-access-8qtbn\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-kube-api-access-8qtbn\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.816276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815499    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istiod-ca-cert\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.816276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815508    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-socket" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:25.816276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815576    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-credential-socket" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:25.816276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815601    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-podinfo\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.816276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815683    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-envoy\") pod \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\" (UID: \"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab\") "
Oct 30 19:58:25.816276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815948    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-certs" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:25.816276 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.815963    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-data" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:25.816619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.816296    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:25.816619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.816319    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:25.816619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.816333    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:25.816619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.816347    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:25.816619 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.816380    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istiod-ca-cert" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:25.818128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.818102    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-kube-api-access-8qtbn" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "kube-api-access-8qtbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:25.818274 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.818247    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-podinfo" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:58:25.818355 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.818322    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-envoy" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:25.818501 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.818484    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-token" pod "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" (UID: "0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:25.822605 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.822242    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"]
Oct 30 19:58:25.917332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.917289    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:25.917332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.917324    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qtbn\" (UniqueName: \"kubernetes.io/projected/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-kube-api-access-8qtbn\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:25.917332 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.917339    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:25.917579 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.917351    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:25.917579 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:25.917365    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:26.418260 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.418228    5232 generic.go:356] "Generic (PLEG): container finished" podID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerID="683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46" exitCode=0
Oct 30 19:58:26.418418 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.418284    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"
Oct 30 19:58:26.418418 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.418313    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" event={"ID":"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab","Type":"ContainerDied","Data":"683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46"}
Oct 30 19:58:26.418418 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.418365    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v" event={"ID":"0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab","Type":"ContainerDied","Data":"0cff19b068614768a44d7aa7f42f58700089d9857a8187db984ee9f4e188ffeb"}
Oct 30 19:58:26.418418 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.418388    5232 scope.go:122] "RemoveContainer" containerID="683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46"
Oct 30 19:58:26.428239 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.428220    5232 scope.go:122] "RemoveContainer" containerID="9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958"
Oct 30 19:58:26.436843 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.436801    5232 scope.go:122] "RemoveContainer" containerID="683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46"
Oct 30 19:58:26.437102 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:26.437080    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46\": container with ID starting with 683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46 not found: ID does not exist" containerID="683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46"
Oct 30 19:58:26.437203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.437107    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46"} err="failed to get container status \"683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46\": rpc error: code = NotFound desc = could not find container \"683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46\": container with ID starting with 683f79b22f75ba15fb5e2220e8914e1a57c384b6ffef74d0f2071e7c899bce46 not found: ID does not exist"
Oct 30 19:58:26.437203 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.437125    5232 scope.go:122] "RemoveContainer" containerID="9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958"
Oct 30 19:58:26.437502 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:26.437480    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958\": container with ID starting with 9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958 not found: ID does not exist" containerID="9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958"
Oct 30 19:58:26.437608 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.437513    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958"} err="failed to get container status \"9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958\": rpc error: code = NotFound desc = could not find container \"9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958\": container with ID starting with 9d0d7566b9acb57157c1fa9a941cb2ca0576a06f110ec6896659bf00831b9958 not found: ID does not exist"
Oct 30 19:58:26.581961 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.581928    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"]
Oct 30 19:58:26.629104 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.629070    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-59594f-openshift-default-66dcf5b9dc-8h44v"]
Oct 30 19:58:26.723839 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.723731    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"]
Oct 30 19:58:26.724296 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.723980    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26" podUID="cae98346-56e2-4e97-ad8f-97b617d91367" containerName="maas-api" containerID="cri-o://c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a" gracePeriod=30
Oct 30 19:58:26.991574 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:26.991474    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:58:27.129970 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.129940    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cae98346-56e2-4e97-ad8f-97b617d91367-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cae98346-56e2-4e97-ad8f-97b617d91367-openshift-service-ca\") pod \"cae98346-56e2-4e97-ad8f-97b617d91367\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") "
Oct 30 19:58:27.130140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.130005    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-metrics-certs\") pod \"cae98346-56e2-4e97-ad8f-97b617d91367\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") "
Oct 30 19:58:27.130140 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.130098    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls\") pod \"cae98346-56e2-4e97-ad8f-97b617d91367\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") "
Oct 30 19:58:27.130280 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.130149    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cae98346-56e2-4e97-ad8f-97b617d91367-kube-api-access-5vj4x\" (UniqueName: \"kubernetes.io/projected/cae98346-56e2-4e97-ad8f-97b617d91367-kube-api-access-5vj4x\") pod \"cae98346-56e2-4e97-ad8f-97b617d91367\" (UID: \"cae98346-56e2-4e97-ad8f-97b617d91367\") "
Oct 30 19:58:27.130366 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.130345    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cae98346-56e2-4e97-ad8f-97b617d91367-openshift-service-ca" pod "cae98346-56e2-4e97-ad8f-97b617d91367" (UID: "cae98346-56e2-4e97-ad8f-97b617d91367"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:27.130671 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.130649    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cae98346-56e2-4e97-ad8f-97b617d91367-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:27.132165 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.132129    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-metrics-certs" pod "cae98346-56e2-4e97-ad8f-97b617d91367" (UID: "cae98346-56e2-4e97-ad8f-97b617d91367"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:27.132317 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.132296    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls" pod "cae98346-56e2-4e97-ad8f-97b617d91367" (UID: "cae98346-56e2-4e97-ad8f-97b617d91367"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:27.132522 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.132498    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae98346-56e2-4e97-ad8f-97b617d91367-kube-api-access-5vj4x" pod "cae98346-56e2-4e97-ad8f-97b617d91367" (UID: "cae98346-56e2-4e97-ad8f-97b617d91367"). InnerVolumeSpecName "kube-api-access-5vj4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:27.232267 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.232237    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:27.232267 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.232263    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cae98346-56e2-4e97-ad8f-97b617d91367-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:27.232466 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.232276    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vj4x\" (UniqueName: \"kubernetes.io/projected/cae98346-56e2-4e97-ad8f-97b617d91367-kube-api-access-5vj4x\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:27.425149 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.425113    5232 generic.go:356] "Generic (PLEG): container finished" podID="cae98346-56e2-4e97-ad8f-97b617d91367" containerID="c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a" exitCode=0
Oct 30 19:58:27.425340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.425159    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"
Oct 30 19:58:27.425340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.425195    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26" event={"ID":"cae98346-56e2-4e97-ad8f-97b617d91367","Type":"ContainerDied","Data":"c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a"}
Oct 30 19:58:27.425340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.425242    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26" event={"ID":"cae98346-56e2-4e97-ad8f-97b617d91367","Type":"ContainerDied","Data":"4cde925fbfd8152354e40984ded49f74e35ab0eaafc75c4856da8b7fe8c80819"}
Oct 30 19:58:27.425340 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.425267    5232 scope.go:122] "RemoveContainer" containerID="c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a"
Oct 30 19:58:27.437892 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.437872    5232 scope.go:122] "RemoveContainer" containerID="c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a"
Oct 30 19:58:27.439233 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:27.438650    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a\": container with ID starting with c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a not found: ID does not exist" containerID="c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a"
Oct 30 19:58:27.439233 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.438689    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a"} err="failed to get container status \"c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a\": rpc error: code = NotFound desc = could not find container \"c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a\": container with ID starting with c43c0614bc67272100dd0fe8e0dc4bfe49039bf279419acb9a3e02dcaa940d9a not found: ID does not exist"
Oct 30 19:58:27.539582 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.539541    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" path="/var/lib/kubelet/pods/0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab/volumes"
Oct 30 19:58:27.586096 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.586062    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"]
Oct 30 19:58:27.596231 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:27.596203    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-d6f58f-78ddd86979-c7v26"]
Oct 30 19:58:29.540015 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:29.539979    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cae98346-56e2-4e97-ad8f-97b617d91367" path="/var/lib/kubelet/pods/cae98346-56e2-4e97-ad8f-97b617d91367/volumes"
Oct 30 19:58:32.835362 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:32.835326    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"]
Oct 30 19:58:32.992657 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:32.992596    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main" containerID="cri-o://a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053" gracePeriod=45
Oct 30 19:58:33.039994 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.039956    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main" containerID="cri-o://a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f" gracePeriod=45
Oct 30 19:58:33.320857 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.320823    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:58:33.362341 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.362310    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:58:33.399756 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.399669    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-model-cache\") pod \"52b5f834-e9aa-4dae-b573-226777de2af1\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") "
Oct 30 19:58:33.399972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.399772    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-tmp-dir\") pod \"52b5f834-e9aa-4dae-b573-226777de2af1\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") "
Oct 30 19:58:33.399972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.399802    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-dshm\") pod \"52b5f834-e9aa-4dae-b573-226777de2af1\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") "
Oct 30 19:58:33.399972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.399837    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52b5f834-e9aa-4dae-b573-226777de2af1-kube-api-access-zp7mj\" (UniqueName: \"kubernetes.io/projected/52b5f834-e9aa-4dae-b573-226777de2af1-kube-api-access-zp7mj\") pod \"52b5f834-e9aa-4dae-b573-226777de2af1\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") "
Oct 30 19:58:33.399972 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.399934    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52b5f834-e9aa-4dae-b573-226777de2af1-tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5f834-e9aa-4dae-b573-226777de2af1-tls-certs\") pod \"52b5f834-e9aa-4dae-b573-226777de2af1\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") "
Oct 30 19:58:33.400218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.399981    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-home\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-home\") pod \"52b5f834-e9aa-4dae-b573-226777de2af1\" (UID: \"52b5f834-e9aa-4dae-b573-226777de2af1\") "
Oct 30 19:58:33.400218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.400014    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-model-cache" pod "52b5f834-e9aa-4dae-b573-226777de2af1" (UID: "52b5f834-e9aa-4dae-b573-226777de2af1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:33.400218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.400047    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-tmp-dir" pod "52b5f834-e9aa-4dae-b573-226777de2af1" (UID: "52b5f834-e9aa-4dae-b573-226777de2af1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:33.400394 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.400300    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-home" pod "52b5f834-e9aa-4dae-b573-226777de2af1" (UID: "52b5f834-e9aa-4dae-b573-226777de2af1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:33.400447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.400417    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.400447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.400430    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.400447 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.400440    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.402240 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.402212    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b5f834-e9aa-4dae-b573-226777de2af1-kube-api-access-zp7mj" pod "52b5f834-e9aa-4dae-b573-226777de2af1" (UID: "52b5f834-e9aa-4dae-b573-226777de2af1"). InnerVolumeSpecName "kube-api-access-zp7mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:33.402336 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.402268    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-dshm" pod "52b5f834-e9aa-4dae-b573-226777de2af1" (UID: "52b5f834-e9aa-4dae-b573-226777de2af1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:33.402336 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.402271    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52b5f834-e9aa-4dae-b573-226777de2af1-tls-certs" pod "52b5f834-e9aa-4dae-b573-226777de2af1" (UID: "52b5f834-e9aa-4dae-b573-226777de2af1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:33.476153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.474535    5232 generic.go:356] "Generic (PLEG): container finished" podID="52b5f834-e9aa-4dae-b573-226777de2af1" containerID="a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053" exitCode=0
Oct 30 19:58:33.476153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.474720    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" event={"ID":"52b5f834-e9aa-4dae-b573-226777de2af1","Type":"ContainerDied","Data":"a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053"}
Oct 30 19:58:33.476153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.474766    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q" event={"ID":"52b5f834-e9aa-4dae-b573-226777de2af1","Type":"ContainerDied","Data":"c2385d6776c88d67abff86a3a7312ee25a7d16b65a984d81ce023468bc200fc9"}
Oct 30 19:58:33.476153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.474790    5232 scope.go:122] "RemoveContainer" containerID="a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053"
Oct 30 19:58:33.476153 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.475019    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"
Oct 30 19:58:33.478860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.477056    5232 generic.go:356] "Generic (PLEG): container finished" podID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerID="a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f" exitCode=0
Oct 30 19:58:33.478860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.477164    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" event={"ID":"47ae7696-22ae-4fee-b71c-5741583f4f80","Type":"ContainerDied","Data":"a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f"}
Oct 30 19:58:33.478860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.477204    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr" event={"ID":"47ae7696-22ae-4fee-b71c-5741583f4f80","Type":"ContainerDied","Data":"9dd30ec83d0d087120009eb84626c9815b66db18f7b44cbb834c262c808ca82e"}
Oct 30 19:58:33.478860 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.477288    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"
Oct 30 19:58:33.491855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.489603    5232 scope.go:122] "RemoveContainer" containerID="a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053"
Oct 30 19:58:33.491855 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:33.490867    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053\": container with ID starting with a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053 not found: ID does not exist" containerID="a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053"
Oct 30 19:58:33.491855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.490913    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053"} err="failed to get container status \"a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053\": rpc error: code = NotFound desc = could not find container \"a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053\": container with ID starting with a8fe20ebcf50b525ab3e5b485ec0f13d6b78c59ba8526eceb541858e8d186053 not found: ID does not exist"
Oct 30 19:58:33.491855 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.490956    5232 scope.go:122] "RemoveContainer" containerID="a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f"
Oct 30 19:58:33.501662 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.501635    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47ae7696-22ae-4fee-b71c-5741583f4f80-kube-api-access-4qcsx\" (UniqueName: \"kubernetes.io/projected/47ae7696-22ae-4fee-b71c-5741583f4f80-kube-api-access-4qcsx\") pod \"47ae7696-22ae-4fee-b71c-5741583f4f80\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") "
Oct 30 19:58:33.501830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.501677    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47ae7696-22ae-4fee-b71c-5741583f4f80-tls-certs\" (UniqueName: \"kubernetes.io/secret/47ae7696-22ae-4fee-b71c-5741583f4f80-tls-certs\") pod \"47ae7696-22ae-4fee-b71c-5741583f4f80\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") "
Oct 30 19:58:33.501830 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.501789    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-model-cache\") pod \"47ae7696-22ae-4fee-b71c-5741583f4f80\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") "
Oct 30 19:58:33.501975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.501830    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-home\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-home\") pod \"47ae7696-22ae-4fee-b71c-5741583f4f80\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") "
Oct 30 19:58:33.501975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.501901    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-tmp-dir\") pod \"47ae7696-22ae-4fee-b71c-5741583f4f80\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") "
Oct 30 19:58:33.502095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502020    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-dshm\") pod \"47ae7696-22ae-4fee-b71c-5741583f4f80\" (UID: \"47ae7696-22ae-4fee-b71c-5741583f4f80\") "
Oct 30 19:58:33.502193 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502162    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-model-cache" pod "47ae7696-22ae-4fee-b71c-5741583f4f80" (UID: "47ae7696-22ae-4fee-b71c-5741583f4f80"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:33.502364 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502330    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-home" pod "47ae7696-22ae-4fee-b71c-5741583f4f80" (UID: "47ae7696-22ae-4fee-b71c-5741583f4f80"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:33.502438 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502370    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-tmp-dir" pod "47ae7696-22ae-4fee-b71c-5741583f4f80" (UID: "47ae7696-22ae-4fee-b71c-5741583f4f80"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:33.502529 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502509    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b5f834-e9aa-4dae-b573-226777de2af1-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.502568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502539    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp7mj\" (UniqueName: \"kubernetes.io/projected/52b5f834-e9aa-4dae-b573-226777de2af1-kube-api-access-zp7mj\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.502568 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502558    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.502644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502573    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.502644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502588    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.502644 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.502603    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b5f834-e9aa-4dae-b573-226777de2af1-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.503195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.503159    5232 scope.go:122] "RemoveContainer" containerID="a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f"
Oct 30 19:58:33.503588 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:33.503557    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f\": container with ID starting with a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f not found: ID does not exist" containerID="a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f"
Oct 30 19:58:33.503694 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.503602    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f"} err="failed to get container status \"a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f\": rpc error: code = NotFound desc = could not find container \"a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f\": container with ID starting with a53f88adc8e7fd2d901e7c629dfefff1c738fed9072a80a1e0505e30ee580c0f not found: ID does not exist"
Oct 30 19:58:33.504118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.504092    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47ae7696-22ae-4fee-b71c-5741583f4f80-kube-api-access-4qcsx" pod "47ae7696-22ae-4fee-b71c-5741583f4f80" (UID: "47ae7696-22ae-4fee-b71c-5741583f4f80"). InnerVolumeSpecName "kube-api-access-4qcsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:33.504325 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.504306    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-dshm" pod "47ae7696-22ae-4fee-b71c-5741583f4f80" (UID: "47ae7696-22ae-4fee-b71c-5741583f4f80"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:33.504642 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.504619    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47ae7696-22ae-4fee-b71c-5741583f4f80-tls-certs" pod "47ae7696-22ae-4fee-b71c-5741583f4f80" (UID: "47ae7696-22ae-4fee-b71c-5741583f4f80"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:33.603975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.603943    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ae7696-22ae-4fee-b71c-5741583f4f80-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.603975 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.603973    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qcsx\" (UniqueName: \"kubernetes.io/projected/47ae7696-22ae-4fee-b71c-5741583f4f80-kube-api-access-4qcsx\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.604244 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.603986    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ae7696-22ae-4fee-b71c-5741583f4f80-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:33.663293 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.663203    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"]
Oct 30 19:58:33.667793 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.667767    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/distinct-2-w3-22fb18-kserve-5688bd6464-4bn4q"]
Oct 30 19:58:33.889409 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.889376    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"]
Oct 30 19:58:33.892512 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:33.892479    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-22fb18/premium-w3-22fb18-kserve-5785fbdd8-x2rbr"]
Oct 30 19:58:35.540126 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:35.540093    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" path="/var/lib/kubelet/pods/47ae7696-22ae-4fee-b71c-5741583f4f80/volumes"
Oct 30 19:58:35.540601 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:35.540506    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" path="/var/lib/kubelet/pods/52b5f834-e9aa-4dae-b573-226777de2af1/volumes"
Oct 30 19:58:37.836070 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:37.836027    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8" podUID="827c3421-c6e8-404f-84b7-69cf1bd1088d" containerName="payload-processing" containerID="cri-o://9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115" gracePeriod=25
Oct 30 19:58:40.841064 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:40.841027    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main" containerID="cri-o://8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d" gracePeriod=45
Oct 30 19:58:40.842147 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:40.842096    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" containerID="cri-o://e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0" gracePeriod=45
Oct 30 19:58:41.007844 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.007094    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main" probeResult="failure" output="Get \"https://10.133.0.85:8000/ready\": dial tcp 10.133.0.85:8000: connect: connection refused"
Oct 30 19:58:41.113715 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.113687    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:58:41.126342 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.126320    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:58:41.261139 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.261103    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8" podUID="827c3421-c6e8-404f-84b7-69cf1bd1088d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:41.290423 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290388    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-model-cache\") pod \"1b361acb-b72d-4eb1-8bd1-272e4399f509\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") "
Oct 30 19:58:41.290615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290451    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-model-cache\") pod \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") "
Oct 30 19:58:41.290615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290485    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-tmp-dir\") pod \"1b361acb-b72d-4eb1-8bd1-272e4399f509\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") "
Oct 30 19:58:41.290615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290521    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tmp-dir\") pod \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") "
Oct 30 19:58:41.290615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290558    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38887470-2bd7-4df2-9aca-cb8d6bccec4a-kube-api-access-r96l4\" (UniqueName: \"kubernetes.io/projected/38887470-2bd7-4df2-9aca-cb8d6bccec4a-kube-api-access-r96l4\") pod \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") "
Oct 30 19:58:41.290615 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290594    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-dshm\") pod \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") "
Oct 30 19:58:41.290941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290766    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-home\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-home\") pod \"1b361acb-b72d-4eb1-8bd1-272e4399f509\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") "
Oct 30 19:58:41.290941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290767    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-model-cache" pod "38887470-2bd7-4df2-9aca-cb8d6bccec4a" (UID: "38887470-2bd7-4df2-9aca-cb8d6bccec4a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:41.290941 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290899    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b361acb-b72d-4eb1-8bd1-272e4399f509-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b361acb-b72d-4eb1-8bd1-272e4399f509-tls-certs\") pod \"1b361acb-b72d-4eb1-8bd1-272e4399f509\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") "
Oct 30 19:58:41.291079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290978    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-home\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-home\") pod \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") "
Oct 30 19:58:41.291079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290772    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-tmp-dir" pod "1b361acb-b72d-4eb1-8bd1-272e4399f509" (UID: "1b361acb-b72d-4eb1-8bd1-272e4399f509"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:41.291079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290927    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tmp-dir" pod "38887470-2bd7-4df2-9aca-cb8d6bccec4a" (UID: "38887470-2bd7-4df2-9aca-cb8d6bccec4a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:41.291079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290982    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-home" pod "1b361acb-b72d-4eb1-8bd1-272e4399f509" (UID: "1b361acb-b72d-4eb1-8bd1-272e4399f509"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:41.291079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291015    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b361acb-b72d-4eb1-8bd1-272e4399f509-kube-api-access-xzb6m\" (UniqueName: \"kubernetes.io/projected/1b361acb-b72d-4eb1-8bd1-272e4399f509-kube-api-access-xzb6m\") pod \"1b361acb-b72d-4eb1-8bd1-272e4399f509\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") "
Oct 30 19:58:41.291079 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291057    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-dshm\") pod \"1b361acb-b72d-4eb1-8bd1-272e4399f509\" (UID: \"1b361acb-b72d-4eb1-8bd1-272e4399f509\") "
Oct 30 19:58:41.291420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291085    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tls-certs\" (UniqueName: \"kubernetes.io/secret/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tls-certs\") pod \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\" (UID: \"38887470-2bd7-4df2-9aca-cb8d6bccec4a\") "
Oct 30 19:58:41.291420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291200    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-home" pod "38887470-2bd7-4df2-9aca-cb8d6bccec4a" (UID: "38887470-2bd7-4df2-9aca-cb8d6bccec4a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:41.291595 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291574    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.291685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291602    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.291685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291617    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.291685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291632    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.291685 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.291647    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.292059 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.290705    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-model-cache" pod "1b361acb-b72d-4eb1-8bd1-272e4399f509" (UID: "1b361acb-b72d-4eb1-8bd1-272e4399f509"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:41.293805 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.293773    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-dshm" pod "1b361acb-b72d-4eb1-8bd1-272e4399f509" (UID: "1b361acb-b72d-4eb1-8bd1-272e4399f509"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:41.293805 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.293796    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38887470-2bd7-4df2-9aca-cb8d6bccec4a-kube-api-access-r96l4" pod "38887470-2bd7-4df2-9aca-cb8d6bccec4a" (UID: "38887470-2bd7-4df2-9aca-cb8d6bccec4a"). InnerVolumeSpecName "kube-api-access-r96l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:41.293805 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.293885    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b361acb-b72d-4eb1-8bd1-272e4399f509-tls-certs" pod "1b361acb-b72d-4eb1-8bd1-272e4399f509" (UID: "1b361acb-b72d-4eb1-8bd1-272e4399f509"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:41.294171 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.294146    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b361acb-b72d-4eb1-8bd1-272e4399f509-kube-api-access-xzb6m" pod "1b361acb-b72d-4eb1-8bd1-272e4399f509" (UID: "1b361acb-b72d-4eb1-8bd1-272e4399f509"). InnerVolumeSpecName "kube-api-access-xzb6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:41.294302 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.294238    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tls-certs" pod "38887470-2bd7-4df2-9aca-cb8d6bccec4a" (UID: "38887470-2bd7-4df2-9aca-cb8d6bccec4a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:41.294483 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.294468    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-dshm" pod "38887470-2bd7-4df2-9aca-cb8d6bccec4a" (UID: "38887470-2bd7-4df2-9aca-cb8d6bccec4a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:41.392699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.392599    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.392699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.392643    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r96l4\" (UniqueName: \"kubernetes.io/projected/38887470-2bd7-4df2-9aca-cb8d6bccec4a-kube-api-access-r96l4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.392699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.392662    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38887470-2bd7-4df2-9aca-cb8d6bccec4a-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.392699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.392676    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b361acb-b72d-4eb1-8bd1-272e4399f509-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.392699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.392689    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzb6m\" (UniqueName: \"kubernetes.io/projected/1b361acb-b72d-4eb1-8bd1-272e4399f509-kube-api-access-xzb6m\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.393109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.392707    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b361acb-b72d-4eb1-8bd1-272e4399f509-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.393109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.392721    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38887470-2bd7-4df2-9aca-cb8d6bccec4a-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:41.526118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.526082    5232 generic.go:356] "Generic (PLEG): container finished" podID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerID="8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d" exitCode=0
Oct 30 19:58:41.526308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.526127    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"
Oct 30 19:58:41.526308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.526175    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" event={"ID":"38887470-2bd7-4df2-9aca-cb8d6bccec4a","Type":"ContainerDied","Data":"8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d"}
Oct 30 19:58:41.526308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.526218    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9" event={"ID":"38887470-2bd7-4df2-9aca-cb8d6bccec4a","Type":"ContainerDied","Data":"d9d62c11b7b5b25c5177b072b2a4321741de5f89d41ee15af98ede2c308a7605"}
Oct 30 19:58:41.526308 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.526236    5232 scope.go:122] "RemoveContainer" containerID="8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d"
Oct 30 19:58:41.527963 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.527942    5232 generic.go:356] "Generic (PLEG): container finished" podID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerID="e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0" exitCode=0
Oct 30 19:58:41.528073 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.528049    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" event={"ID":"1b361acb-b72d-4eb1-8bd1-272e4399f509","Type":"ContainerDied","Data":"e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0"}
Oct 30 19:58:41.528120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.528071    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g" event={"ID":"1b361acb-b72d-4eb1-8bd1-272e4399f509","Type":"ContainerDied","Data":"d8847a26e8eaaceb1482d5f737c05fbd1534d5e4ff4b2c05177633864d5c9c3c"}
Oct 30 19:58:41.528120 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.528115    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"
Oct 30 19:58:41.537416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.537388    5232 scope.go:122] "RemoveContainer" containerID="8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d"
Oct 30 19:58:41.537742 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:41.537723    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d\": container with ID starting with 8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d not found: ID does not exist" containerID="8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d"
Oct 30 19:58:41.537853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.537750    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d"} err="failed to get container status \"8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d\": rpc error: code = NotFound desc = could not find container \"8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d\": container with ID starting with 8642f13ffa0f5c43523e991839c4375f0345ebb97710c7d49a371dba7d64943d not found: ID does not exist"
Oct 30 19:58:41.537853 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.537769    5232 scope.go:122] "RemoveContainer" containerID="e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0"
Oct 30 19:58:41.548136 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.548115    5232 scope.go:122] "RemoveContainer" containerID="e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0"
Oct 30 19:58:41.548399 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:41.548376    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0\": container with ID starting with e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0 not found: ID does not exist" containerID="e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0"
Oct 30 19:58:41.548477 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.548410    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0"} err="failed to get container status \"e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0\": rpc error: code = NotFound desc = could not find container \"e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0\": container with ID starting with e067bdc800a262aaa473e3a06868869a6143e0f36744dbb962232c9f0831faf0 not found: ID does not exist"
Oct 30 19:58:41.686235 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.686144    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"]
Oct 30 19:58:41.690577 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.690551    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-2-w1-9275c3-kserve-85bb6668bb-zkns9"]
Oct 30 19:58:41.798722 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.798686    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"]
Oct 30 19:58:41.804420 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:41.804394    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-9275c3/distinct-w1-9275c3-kserve-6db7d485d5-gbg2g"]
Oct 30 19:58:42.185535 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:42.185490    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"]
Oct 30 19:58:43.539248 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:43.539215    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" path="/var/lib/kubelet/pods/1b361acb-b72d-4eb1-8bd1-272e4399f509/volumes"
Oct 30 19:58:43.539673 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:43.539655    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" path="/var/lib/kubelet/pods/38887470-2bd7-4df2-9aca-cb8d6bccec4a/volumes"
Oct 30 19:58:46.187805 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:46.187768    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:58:48.855530 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.855491    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"]
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856050    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856066    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856084    5232 cpu_manager.go:414] "containerMap: removing container" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856091    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856105    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856111    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856116    5232 cpu_manager.go:414] "containerMap: removing container" podUID="519960a2-3a77-4228-8f22-0455fcd3cfa9" containerName="maas-api"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856121    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="519960a2-3a77-4228-8f22-0455fcd3cfa9" containerName="maas-api"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856127    5232 cpu_manager.go:414] "containerMap: removing container" podUID="525f236d-76e8-4404-9c8c-5af6416906de" containerName="maas-api"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856132    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="525f236d-76e8-4404-9c8c-5af6416906de" containerName="maas-api"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856139    5232 cpu_manager.go:414] "containerMap: removing container" podUID="9dbdae8e-cabf-4056-8f45-550d43dafa5f" containerName="maas-api"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856143    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dbdae8e-cabf-4056-8f45-550d43dafa5f" containerName="maas-api"
Oct 30 19:58:48.856144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856153    5232 cpu_manager.go:414] "containerMap: removing container" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856160    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856180    5232 cpu_manager.go:414] "containerMap: removing container" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856185    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856192    5232 cpu_manager.go:414] "containerMap: removing container" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856197    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856204    5232 cpu_manager.go:414] "containerMap: removing container" podUID="330a6418-9381-4c85-a56c-5e238567f9f0" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856208    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="330a6418-9381-4c85-a56c-5e238567f9f0" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856215    5232 cpu_manager.go:414] "containerMap: removing container" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856220    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856229    5232 cpu_manager.go:414] "containerMap: removing container" podUID="a51573e1-e9cb-44ef-94d7-c25d887678a4" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856233    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a51573e1-e9cb-44ef-94d7-c25d887678a4" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856244    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856251    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856265    5232 cpu_manager.go:414] "containerMap: removing container" podUID="cae98346-56e2-4e97-ad8f-97b617d91367" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856271    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cae98346-56e2-4e97-ad8f-97b617d91367" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856278    5232 cpu_manager.go:414] "containerMap: removing container" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856283    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856291    5232 cpu_manager.go:414] "containerMap: removing container" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856295    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856301    5232 cpu_manager.go:414] "containerMap: removing container" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856306    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856313    5232 cpu_manager.go:414] "containerMap: removing container" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856318    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856330    5232 cpu_manager.go:414] "containerMap: removing container" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856336    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856417    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856425    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="52b5f834-e9aa-4dae-b573-226777de2af1" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856432    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856438    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="519960a2-3a77-4228-8f22-0455fcd3cfa9" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856444    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="525f236d-76e8-4404-9c8c-5af6416906de" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856451    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dbdae8e-cabf-4056-8f45-550d43dafa5f" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856457    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="38887470-2bd7-4df2-9aca-cb8d6bccec4a" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856462    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856469    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9c7681e-81cf-4bdb-b23b-004d3a614538" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856474    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856483    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="330a6418-9381-4c85-a56c-5e238567f9f0" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856491    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="5401cf32-a2fd-4804-b703-5cf2116c6510" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856498    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="a51573e1-e9cb-44ef-94d7-c25d887678a4" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856505    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="cae98346-56e2-4e97-ad8f-97b617d91367" containerName="maas-api"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856511    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856517    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="78cb9d9c-5179-4dd3-a31f-a71b9bff4f38" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856524    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="494821ca-b40d-4423-b192-0ce5a101eeb4" containerName="istio-proxy"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856530    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="47ae7696-22ae-4fee-b71c-5741583f4f80" containerName="main"
Oct 30 19:58:48.856748 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.856536    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b361acb-b72d-4eb1-8bd1-272e4399f509" containerName="main"
Oct 30 19:58:48.861856 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.861837    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:48.914796 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.914755    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-f77d05-serving-cert\""
Oct 30 19:58:48.964689 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.964654    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"]
Oct 30 19:58:48.968971 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.968938    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njspz\" (UniqueName: \"kubernetes.io/projected/23f11210-07cb-4964-a8e6-5364f05adff8-kube-api-access-njspz\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:48.969128 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.969084    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:48.969218 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.969193    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-metrics-certs\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:48.969269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:48.969252    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f11210-07cb-4964-a8e6-5364f05adff8-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.070379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.070331    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.070379 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.070387    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-metrics-certs\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.070640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.070407    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f11210-07cb-4964-a8e6-5364f05adff8-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.070640 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:49.070478    5232 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-serving-cert: secret "maas-api-e2e-extmod-b-f77d05-serving-cert" not found
Oct 30 19:58:49.070640 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:49.070544    5232 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls podName:23f11210-07cb-4964-a8e6-5364f05adff8 nodeName:}" failed. No retries permitted until 2026-10-30 19:58:49.570524434 +0000 UTC m=+4322.155007408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls") pod "maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d" (UID: "23f11210-07cb-4964-a8e6-5364f05adff8") : secret "maas-api-e2e-extmod-b-f77d05-serving-cert" not found
Oct 30 19:58:49.070640 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.070596    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njspz\" (UniqueName: \"kubernetes.io/projected/23f11210-07cb-4964-a8e6-5364f05adff8-kube-api-access-njspz\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.071142 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.071124    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f11210-07cb-4964-a8e6-5364f05adff8-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.072995 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.072973    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-metrics-certs\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.140230 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.140154    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njspz\" (UniqueName: \"kubernetes.io/projected/23f11210-07cb-4964-a8e6-5364f05adff8-kube-api-access-njspz\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.576445 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.576362    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.578836 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.578794    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:49.775916 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:49.775883    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:50.153295 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:50.153255    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"]
Oct 30 19:58:50.160512 ip-10-0-143-248 kubenswrapper[5232]: W1030 19:58:50.160485    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23f11210_07cb_4964_a8e6_5364f05adff8.slice/crio-c9dd24c8f44096c7ee32d89e3b1ebfa6062f9e0d165d239a7273ab162d36878a WatchSource:0}: Error finding container c9dd24c8f44096c7ee32d89e3b1ebfa6062f9e0d165d239a7273ab162d36878a: Status 404 returned error can't find the container with id c9dd24c8f44096c7ee32d89e3b1ebfa6062f9e0d165d239a7273ab162d36878a
Oct 30 19:58:50.161834 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:50.161801    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:58:50.589460 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:50.589424    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d" event={"ID":"23f11210-07cb-4964-a8e6-5364f05adff8","Type":"ContainerStarted","Data":"c9dd24c8f44096c7ee32d89e3b1ebfa6062f9e0d165d239a7273ab162d36878a"}
Oct 30 19:58:51.260428 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:51.260400    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8" podUID="827c3421-c6e8-404f-84b7-69cf1bd1088d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:51.358450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:51.358416    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"]
Oct 30 19:58:51.597076 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:51.597040    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d" event={"ID":"23f11210-07cb-4964-a8e6-5364f05adff8","Type":"ContainerStarted","Data":"b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124"}
Oct 30 19:58:51.597269 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:51.597102    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:51.712730 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:51.712675    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d" podStartSLOduration=2.685438759 podStartE2EDuration="3.712662673s" totalImagesPullingTime="1.027223914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:51.711305318 +0000 UTC m=+4324.295788315" watchObservedRunningTime="2026-10-30 19:58:51.712662673 +0000 UTC m=+4324.297145669"
Oct 30 19:58:56.359503 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:56.359409    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2" podUID="56f9b6fd-b4e9-495c-bafe-71522b3d3469" containerName="payload-processing" containerID="cri-o://3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd" gracePeriod=25
Oct 30 19:58:57.205496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.205436    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main" containerID="cri-o://de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae" gracePeriod=45
Oct 30 19:58:57.460669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.460606    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:58:57.556123 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556090    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-home\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-home\") pod \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") "
Oct 30 19:58:57.556294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556189    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tls-certs\") pod \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") "
Oct 30 19:58:57.556294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556212    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-dshm\") pod \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") "
Oct 30 19:58:57.556294 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556231    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-model-cache\") pod \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") "
Oct 30 19:58:57.556438 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556297    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9b68dad-e4e3-4892-924a-2cb5c605fa18-kube-api-access-pqhdt\" (UniqueName: \"kubernetes.io/projected/f9b68dad-e4e3-4892-924a-2cb5c605fa18-kube-api-access-pqhdt\") pod \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") "
Oct 30 19:58:57.556438 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556332    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tmp-dir\") pod \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\" (UID: \"f9b68dad-e4e3-4892-924a-2cb5c605fa18\") "
Oct 30 19:58:57.556438 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556346    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-home" pod "f9b68dad-e4e3-4892-924a-2cb5c605fa18" (UID: "f9b68dad-e4e3-4892-924a-2cb5c605fa18"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.556607 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556560    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-model-cache" pod "f9b68dad-e4e3-4892-924a-2cb5c605fa18" (UID: "f9b68dad-e4e3-4892-924a-2cb5c605fa18"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.556669 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556613    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tmp-dir" pod "f9b68dad-e4e3-4892-924a-2cb5c605fa18" (UID: "f9b68dad-e4e3-4892-924a-2cb5c605fa18"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.556782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556745    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.556782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556769    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.556782 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.556783    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.558450 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.558429    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-dshm" pod "f9b68dad-e4e3-4892-924a-2cb5c605fa18" (UID: "f9b68dad-e4e3-4892-924a-2cb5c605fa18"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.558545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.558488    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tls-certs" pod "f9b68dad-e4e3-4892-924a-2cb5c605fa18" (UID: "f9b68dad-e4e3-4892-924a-2cb5c605fa18"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:57.558545 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.558525    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b68dad-e4e3-4892-924a-2cb5c605fa18-kube-api-access-pqhdt" pod "f9b68dad-e4e3-4892-924a-2cb5c605fa18" (UID: "f9b68dad-e4e3-4892-924a-2cb5c605fa18"). InnerVolumeSpecName "kube-api-access-pqhdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:57.610118 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.610080    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 19:58:57.636037 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.636004    5232 generic.go:356] "Generic (PLEG): container finished" podID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerID="de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae" exitCode=0
Oct 30 19:58:57.636325 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.636062    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" event={"ID":"f9b68dad-e4e3-4892-924a-2cb5c605fa18","Type":"ContainerDied","Data":"de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae"}
Oct 30 19:58:57.636424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.636350    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p" event={"ID":"f9b68dad-e4e3-4892-924a-2cb5c605fa18","Type":"ContainerDied","Data":"e6af518369878699cd3894c0ca426cd2c46143554b5ae74d90f7e220f19a91d1"}
Oct 30 19:58:57.636424 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.636380    5232 scope.go:122] "RemoveContainer" containerID="de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae"
Oct 30 19:58:57.636544 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.636078    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"
Oct 30 19:58:57.647497 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.647471    5232 scope.go:122] "RemoveContainer" containerID="de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae"
Oct 30 19:58:57.647847 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:57.647807    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae\": container with ID starting with de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae not found: ID does not exist" containerID="de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae"
Oct 30 19:58:57.647977 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.647858    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae"} err="failed to get container status \"de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae\": rpc error: code = NotFound desc = could not find container \"de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae\": container with ID starting with de2e4717a4ff3765ef6fb0bd61c8bdd90ce3b61902a3ead1c5f6baf0265305ae not found: ID does not exist"
Oct 30 19:58:57.657904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.657873    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9b68dad-e4e3-4892-924a-2cb5c605fa18-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.657904 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.657903    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9b68dad-e4e3-4892-924a-2cb5c605fa18-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.658119 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.657918    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqhdt\" (UniqueName: \"kubernetes.io/projected/f9b68dad-e4e3-4892-924a-2cb5c605fa18-kube-api-access-pqhdt\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.842960 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.842914    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"]
Oct 30 19:58:57.845683 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:57.845375    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/premium-w0-2301cb-kserve-676d6d78d5-kj64p"]
Oct 30 19:58:58.079487 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.079463    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:58:58.165163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.163413    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/827c3421-c6e8-404f-84b7-69cf1bd1088d-kube-api-access-fxj9g\" (UniqueName: \"kubernetes.io/projected/827c3421-c6e8-404f-84b7-69cf1bd1088d-kube-api-access-fxj9g\") pod \"827c3421-c6e8-404f-84b7-69cf1bd1088d\" (UID: \"827c3421-c6e8-404f-84b7-69cf1bd1088d\") "
Oct 30 19:58:58.165163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.163503    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/827c3421-c6e8-404f-84b7-69cf1bd1088d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/827c3421-c6e8-404f-84b7-69cf1bd1088d-plugins-config-volume\") pod \"827c3421-c6e8-404f-84b7-69cf1bd1088d\" (UID: \"827c3421-c6e8-404f-84b7-69cf1bd1088d\") "
Oct 30 19:58:58.165163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.163993    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/827c3421-c6e8-404f-84b7-69cf1bd1088d-plugins-config-volume" pod "827c3421-c6e8-404f-84b7-69cf1bd1088d" (UID: "827c3421-c6e8-404f-84b7-69cf1bd1088d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:58.165890 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.165871    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/827c3421-c6e8-404f-84b7-69cf1bd1088d-kube-api-access-fxj9g" pod "827c3421-c6e8-404f-84b7-69cf1bd1088d" (UID: "827c3421-c6e8-404f-84b7-69cf1bd1088d"). InnerVolumeSpecName "kube-api-access-fxj9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:58.266135 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.266100    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxj9g\" (UniqueName: \"kubernetes.io/projected/827c3421-c6e8-404f-84b7-69cf1bd1088d-kube-api-access-fxj9g\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:58.266135 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.266135    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/827c3421-c6e8-404f-84b7-69cf1bd1088d-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:58.645025 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.644993    5232 generic.go:356] "Generic (PLEG): container finished" podID="827c3421-c6e8-404f-84b7-69cf1bd1088d" containerID="9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115" exitCode=0
Oct 30 19:58:58.645433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.645042    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"
Oct 30 19:58:58.645433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.645049    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8" event={"ID":"827c3421-c6e8-404f-84b7-69cf1bd1088d","Type":"ContainerDied","Data":"9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115"}
Oct 30 19:58:58.645433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.645164    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8" event={"ID":"827c3421-c6e8-404f-84b7-69cf1bd1088d","Type":"ContainerDied","Data":"31ec280a77584367b48c650a2bd7c1162d5a15a8c7f6e1a312f7efa52ea4380b"}
Oct 30 19:58:58.645433 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.645191    5232 scope.go:122] "RemoveContainer" containerID="9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115"
Oct 30 19:58:58.655061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.655046    5232 scope.go:122] "RemoveContainer" containerID="9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115"
Oct 30 19:58:58.655319 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:58:58.655300    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115\": container with ID starting with 9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115 not found: ID does not exist" containerID="9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115"
Oct 30 19:58:58.655372 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.655327    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115"} err="failed to get container status \"9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115\": rpc error: code = NotFound desc = could not find container \"9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115\": container with ID starting with 9b7eddfaf438e6e3d9ad0221bee22667814cbed15567be020a67f2a9a7538115 not found: ID does not exist"
Oct 30 19:58:58.807882 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.807838    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"]
Oct 30 19:58:58.812667 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:58.812635    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-9275c3-67c46bf4db-kk8d8"]
Oct 30 19:58:59.539973 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:59.539937    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="827c3421-c6e8-404f-84b7-69cf1bd1088d" path="/var/lib/kubelet/pods/827c3421-c6e8-404f-84b7-69cf1bd1088d/volumes"
Oct 30 19:58:59.540416 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:58:59.540390    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" path="/var/lib/kubelet/pods/f9b68dad-e4e3-4892-924a-2cb5c605fa18/volumes"
Oct 30 19:59:03.023460 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.023424    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"]
Oct 30 19:59:03.023997 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.023696    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4" podUID="7d447874-7803-46a2-bcbe-e8a9bffb33d4" containerName="maas-api" containerID="cri-o://5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85" gracePeriod=30
Oct 30 19:59:03.287656 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.287589    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:59:03.418246 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.418203    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-metrics-certs\") pod \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") "
Oct 30 19:59:03.418405 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.418278    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d447874-7803-46a2-bcbe-e8a9bffb33d4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d447874-7803-46a2-bcbe-e8a9bffb33d4-openshift-service-ca\") pod \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") "
Oct 30 19:59:03.418405 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.418309    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls\") pod \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") "
Oct 30 19:59:03.418507 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.418418    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d447874-7803-46a2-bcbe-e8a9bffb33d4-kube-api-access-spsj9\" (UniqueName: \"kubernetes.io/projected/7d447874-7803-46a2-bcbe-e8a9bffb33d4-kube-api-access-spsj9\") pod \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\" (UID: \"7d447874-7803-46a2-bcbe-e8a9bffb33d4\") "
Oct 30 19:59:03.418948 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.418903    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d447874-7803-46a2-bcbe-e8a9bffb33d4-openshift-service-ca" pod "7d447874-7803-46a2-bcbe-e8a9bffb33d4" (UID: "7d447874-7803-46a2-bcbe-e8a9bffb33d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:03.421165 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.421122    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d447874-7803-46a2-bcbe-e8a9bffb33d4-kube-api-access-spsj9" pod "7d447874-7803-46a2-bcbe-e8a9bffb33d4" (UID: "7d447874-7803-46a2-bcbe-e8a9bffb33d4"). InnerVolumeSpecName "kube-api-access-spsj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:03.421298 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.421243    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls" pod "7d447874-7803-46a2-bcbe-e8a9bffb33d4" (UID: "7d447874-7803-46a2-bcbe-e8a9bffb33d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:03.421412 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.421385    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-metrics-certs" pod "7d447874-7803-46a2-bcbe-e8a9bffb33d4" (UID: "7d447874-7803-46a2-bcbe-e8a9bffb33d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:03.519515 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.519466    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spsj9\" (UniqueName: \"kubernetes.io/projected/7d447874-7803-46a2-bcbe-e8a9bffb33d4-kube-api-access-spsj9\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:03.519515 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.519507    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:03.519515 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.519521    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d447874-7803-46a2-bcbe-e8a9bffb33d4-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:03.519772 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.519534    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d447874-7803-46a2-bcbe-e8a9bffb33d4-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:03.677617 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.677584    5232 generic.go:356] "Generic (PLEG): container finished" podID="7d447874-7803-46a2-bcbe-e8a9bffb33d4" containerID="5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85" exitCode=0
Oct 30 19:59:03.677954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.677639    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"
Oct 30 19:59:03.677954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.677675    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4" event={"ID":"7d447874-7803-46a2-bcbe-e8a9bffb33d4","Type":"ContainerDied","Data":"5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85"}
Oct 30 19:59:03.677954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.677736    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4" event={"ID":"7d447874-7803-46a2-bcbe-e8a9bffb33d4","Type":"ContainerDied","Data":"b5e340df5f76c2e39b6bb0ae0f5df5ebc30324011d1d7db12658514718c74aa6"}
Oct 30 19:59:03.677954 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.677765    5232 scope.go:122] "RemoveContainer" containerID="5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85"
Oct 30 19:59:03.687695 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.687674    5232 scope.go:122] "RemoveContainer" containerID="5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85"
Oct 30 19:59:03.688069 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:59:03.688038    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85\": container with ID starting with 5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85 not found: ID does not exist" containerID="5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85"
Oct 30 19:59:03.688144 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.688082    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85"} err="failed to get container status \"5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85\": rpc error: code = NotFound desc = could not find container \"5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85\": container with ID starting with 5395c9fcaa92db1fdc5f0d312e70fca88ded5924f8e8a32146fc2f6e024bcb85 not found: ID does not exist"
Oct 30 19:59:03.837767 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.837733    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"]
Oct 30 19:59:03.846255 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:03.846223    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4"]
Oct 30 19:59:05.539629 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:05.539594    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d447874-7803-46a2-bcbe-e8a9bffb33d4" path="/var/lib/kubelet/pods/7d447874-7803-46a2-bcbe-e8a9bffb33d4/volumes"
Oct 30 19:59:05.585699 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:05.585657    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2" podUID="56f9b6fd-b4e9-495c-bafe-71522b3d3469" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:59:06.201309 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:06.201269    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-85f471-86c7965f9b-n99x4" podUID="7d447874-7803-46a2-bcbe-e8a9bffb33d4" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.153:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:59:15.586237 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:15.586193    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2" podUID="56f9b6fd-b4e9-495c-bafe-71522b3d3469" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:59:16.187692 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:59:16.187641    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:59:16.613387 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.613359    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:59:16.757798 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.757692    5232 generic.go:356] "Generic (PLEG): container finished" podID="56f9b6fd-b4e9-495c-bafe-71522b3d3469" containerID="3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd" exitCode=0
Oct 30 19:59:16.757798 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.757744    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"
Oct 30 19:59:16.758033 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.757743    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2" event={"ID":"56f9b6fd-b4e9-495c-bafe-71522b3d3469","Type":"ContainerDied","Data":"3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd"}
Oct 30 19:59:16.758033 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.757913    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2" event={"ID":"56f9b6fd-b4e9-495c-bafe-71522b3d3469","Type":"ContainerDied","Data":"cf410e48daf73c729e6ae0b0a0bed21dff2d62e91b71208ad2b015fa4905f771"}
Oct 30 19:59:16.758033 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.757946    5232 scope.go:122] "RemoveContainer" containerID="3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd"
Oct 30 19:59:16.770109 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.770052    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56f9b6fd-b4e9-495c-bafe-71522b3d3469-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56f9b6fd-b4e9-495c-bafe-71522b3d3469-plugins-config-volume\") pod \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\" (UID: \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\") "
Oct 30 19:59:16.770299 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.770263    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56f9b6fd-b4e9-495c-bafe-71522b3d3469-kube-api-access-g7xbm\" (UniqueName: \"kubernetes.io/projected/56f9b6fd-b4e9-495c-bafe-71522b3d3469-kube-api-access-g7xbm\") pod \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\" (UID: \"56f9b6fd-b4e9-495c-bafe-71522b3d3469\") "
Oct 30 19:59:16.772820 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.770550    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56f9b6fd-b4e9-495c-bafe-71522b3d3469-plugins-config-volume" pod "56f9b6fd-b4e9-495c-bafe-71522b3d3469" (UID: "56f9b6fd-b4e9-495c-bafe-71522b3d3469"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:16.773032 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.772917    5232 scope.go:122] "RemoveContainer" containerID="3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd"
Oct 30 19:59:16.773032 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.773000    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56f9b6fd-b4e9-495c-bafe-71522b3d3469-kube-api-access-g7xbm" pod "56f9b6fd-b4e9-495c-bafe-71522b3d3469" (UID: "56f9b6fd-b4e9-495c-bafe-71522b3d3469"). InnerVolumeSpecName "kube-api-access-g7xbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:16.774158 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:59:16.774128    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd\": container with ID starting with 3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd not found: ID does not exist" containerID="3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd"
Oct 30 19:59:16.774251 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.774165    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd"} err="failed to get container status \"3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd\": rpc error: code = NotFound desc = could not find container \"3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd\": container with ID starting with 3879dea962d62093ad60140cfdf340b4d61311b6fae1b16f8a7c5fa6b53b74cd not found: ID does not exist"
Oct 30 19:59:16.871108 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.871070    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7xbm\" (UniqueName: \"kubernetes.io/projected/56f9b6fd-b4e9-495c-bafe-71522b3d3469-kube-api-access-g7xbm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:16.871108 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:16.871102    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56f9b6fd-b4e9-495c-bafe-71522b3d3469-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:17.183803 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:17.183766    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"]
Oct 30 19:59:17.231360 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:17.231329    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2301cb-69f64cbf4d-h2km2"]
Oct 30 19:59:17.539913 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:17.539879    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56f9b6fd-b4e9-495c-bafe-71522b3d3469" path="/var/lib/kubelet/pods/56f9b6fd-b4e9-495c-bafe-71522b3d3469/volumes"
Oct 30 19:59:36.578826 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.578784    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"]
Oct 30 19:59:36.581305 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.579011    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n" podUID="198f0b29-1953-4a6b-88ee-638cd1f6e114" containerName="maas-api" containerID="cri-o://3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077" gracePeriod=30
Oct 30 19:59:36.849320 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.849292    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:59:36.898571 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.898537    5232 generic.go:356] "Generic (PLEG): container finished" podID="198f0b29-1953-4a6b-88ee-638cd1f6e114" containerID="3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077" exitCode=0
Oct 30 19:59:36.898743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.898586    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"
Oct 30 19:59:36.898743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.898592    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n" event={"ID":"198f0b29-1953-4a6b-88ee-638cd1f6e114","Type":"ContainerDied","Data":"3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077"}
Oct 30 19:59:36.898743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.898644    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n" event={"ID":"198f0b29-1953-4a6b-88ee-638cd1f6e114","Type":"ContainerDied","Data":"6e6e23f80f3706aca0529b4bce1a70595f34cf943516daf897578132344a9a55"}
Oct 30 19:59:36.898743 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.898674    5232 scope.go:122] "RemoveContainer" containerID="3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077"
Oct 30 19:59:36.908184 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.908166    5232 scope.go:122] "RemoveContainer" containerID="3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077"
Oct 30 19:59:36.908471 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:59:36.908447    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077\": container with ID starting with 3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077 not found: ID does not exist" containerID="3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077"
Oct 30 19:59:36.908529 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.908478    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077"} err="failed to get container status \"3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077\": rpc error: code = NotFound desc = could not find container \"3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077\": container with ID starting with 3a00a52282f19398cd12a978e4ecc9688c6b76f8485f933ef89c1df93c8a2077 not found: ID does not exist"
Oct 30 19:59:36.979238 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.979208    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/198f0b29-1953-4a6b-88ee-638cd1f6e114-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/198f0b29-1953-4a6b-88ee-638cd1f6e114-openshift-service-ca\") pod \"198f0b29-1953-4a6b-88ee-638cd1f6e114\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") "
Oct 30 19:59:36.979430 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.979271    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-metrics-certs\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-metrics-certs\") pod \"198f0b29-1953-4a6b-88ee-638cd1f6e114\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") "
Oct 30 19:59:36.979511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.979468    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls\") pod \"198f0b29-1953-4a6b-88ee-638cd1f6e114\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") "
Oct 30 19:59:36.979511 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.979503    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/198f0b29-1953-4a6b-88ee-638cd1f6e114-kube-api-access-jv8ln\" (UniqueName: \"kubernetes.io/projected/198f0b29-1953-4a6b-88ee-638cd1f6e114-kube-api-access-jv8ln\") pod \"198f0b29-1953-4a6b-88ee-638cd1f6e114\" (UID: \"198f0b29-1953-4a6b-88ee-638cd1f6e114\") "
Oct 30 19:59:36.979638 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.979592    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198f0b29-1953-4a6b-88ee-638cd1f6e114-openshift-service-ca" pod "198f0b29-1953-4a6b-88ee-638cd1f6e114" (UID: "198f0b29-1953-4a6b-88ee-638cd1f6e114"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:36.979907 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.979883    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/198f0b29-1953-4a6b-88ee-638cd1f6e114-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:36.981757 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.981727    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198f0b29-1953-4a6b-88ee-638cd1f6e114-kube-api-access-jv8ln" pod "198f0b29-1953-4a6b-88ee-638cd1f6e114" (UID: "198f0b29-1953-4a6b-88ee-638cd1f6e114"). InnerVolumeSpecName "kube-api-access-jv8ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:36.982048 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.982017    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls" pod "198f0b29-1953-4a6b-88ee-638cd1f6e114" (UID: "198f0b29-1953-4a6b-88ee-638cd1f6e114"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:36.982048 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:36.982036    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-metrics-certs" pod "198f0b29-1953-4a6b-88ee-638cd1f6e114" (UID: "198f0b29-1953-4a6b-88ee-638cd1f6e114"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:37.091377 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:37.080599    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:37.091377 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:37.080636    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv8ln\" (UniqueName: \"kubernetes.io/projected/198f0b29-1953-4a6b-88ee-638cd1f6e114-kube-api-access-jv8ln\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:37.091377 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:37.080652    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/198f0b29-1953-4a6b-88ee-638cd1f6e114-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:37.316072 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:37.316038    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"]
Oct 30 19:59:37.320110 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:37.320080    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-9275c3-557c4679d6-t6q9n"]
Oct 30 19:59:37.539062 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:37.538985    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="198f0b29-1953-4a6b-88ee-638cd1f6e114" path="/var/lib/kubelet/pods/198f0b29-1953-4a6b-88ee-638cd1f6e114/volumes"
Oct 30 19:59:46.187623 ip-10-0-143-248 kubenswrapper[5232]: E1030 19:59:46.187583    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 19:59:46.751985 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:46.751955    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"]
Oct 30 19:59:46.752264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:46.752230    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy" containerID="cri-o://3daf776f6b0a58a65e458e9640ad02bef3d91ab785be57b052f6872c4ae30eee" gracePeriod=30
Oct 30 19:59:47.992727 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:47.992687    5232 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:59:47.993163 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:47.992754    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:59:51.994264 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:51.994227    5232 generic.go:356] "Generic (PLEG): container finished" podID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerID="3daf776f6b0a58a65e458e9640ad02bef3d91ab785be57b052f6872c4ae30eee" exitCode=0
Oct 30 19:59:51.994609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:51.994297    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" event={"ID":"e69a2eb3-19ad-417f-854e-49b0c11167a6","Type":"ContainerDied","Data":"3daf776f6b0a58a65e458e9640ad02bef3d91ab785be57b052f6872c4ae30eee"}
Oct 30 19:59:51.994609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:51.994373    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc" event={"ID":"e69a2eb3-19ad-417f-854e-49b0c11167a6","Type":"ContainerDied","Data":"453ffe061c3b518aacde16f3226a385be8357a21ad7de1f68f4eac70bb2a75b7"}
Oct 30 19:59:51.994609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:51.994398    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="453ffe061c3b518aacde16f3226a385be8357a21ad7de1f68f4eac70bb2a75b7"
Oct 30 19:59:51.994609 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:51.994416    5232 scope.go:122] "RemoveContainer" containerID="aa3cd0ffb061e801d5978981a2bf46a8b1003951550099b12bd750fcedbd3401"
Oct 30 19:59:52.010946 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.010921    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:59:52.130636 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130602    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-token\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-token\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.130852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130646    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-socket\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.130852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130672    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-podinfo\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.130852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130693    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-data\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.130852 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130824    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-credential-socket\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.131095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130859    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-kube-api-access-gmr49\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-kube-api-access-gmr49\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.131095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130904    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-envoy\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.131095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130928    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e69a2eb3-19ad-417f-854e-49b0c11167a6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e69a2eb3-19ad-417f-854e-49b0c11167a6-istiod-ca-cert\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.131095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.130978    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-certs\") pod \"e69a2eb3-19ad-417f-854e-49b0c11167a6\" (UID: \"e69a2eb3-19ad-417f-854e-49b0c11167a6\") "
Oct 30 19:59:52.131095 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131047    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-socket" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.131402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131231    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-credential-socket" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.131402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131280    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-certs" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.131402 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131299    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-data" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.131510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131439    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.131510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131460    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.131510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131474    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.131510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131484    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e69a2eb3-19ad-417f-854e-49b0c11167a6-istiod-ca-cert" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:52.131510 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.131488    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.133071 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.133042    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-envoy" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.133195 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.133162    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-podinfo" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:59:52.133326 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.133310    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-token" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:52.133394 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.133371    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-kube-api-access-gmr49" pod "e69a2eb3-19ad-417f-854e-49b0c11167a6" (UID: "e69a2eb3-19ad-417f-854e-49b0c11167a6"). InnerVolumeSpecName "kube-api-access-gmr49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:52.232788 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.232750    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmr49\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-kube-api-access-gmr49\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.232788 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.232785    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.232788 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.232794    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e69a2eb3-19ad-417f-854e-49b0c11167a6-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.233074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.232802    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.233074 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:52.232834    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e69a2eb3-19ad-417f-854e-49b0c11167a6-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:53.002175 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:53.002141    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"
Oct 30 19:59:53.279808 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:53.279721    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"]
Oct 30 19:59:53.285562 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:53.285534    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-dwhtc"]
Oct 30 19:59:53.538876 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:53.538771    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" path="/var/lib/kubelet/pods/e69a2eb3-19ad-417f-854e-49b0c11167a6/volumes"
Oct 30 19:59:57.234061 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:57.234028    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"]
Oct 30 19:59:57.234496 ip-10-0-143-248 kubenswrapper[5232]: I1030 19:59:57.234319    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy" containerID="cri-o://a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82" gracePeriod=30
Oct 30 20:00:00.178090 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178054    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"]
Oct 30 20:00:00.178505 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178464    5232 cpu_manager.go:414] "containerMap: removing container" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy"
Oct 30 20:00:00.178505 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178475    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy"
Oct 30 20:00:00.178505 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178485    5232 cpu_manager.go:414] "containerMap: removing container" podUID="56f9b6fd-b4e9-495c-bafe-71522b3d3469" containerName="payload-processing"
Oct 30 20:00:00.178505 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178490    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56f9b6fd-b4e9-495c-bafe-71522b3d3469" containerName="payload-processing"
Oct 30 20:00:00.178505 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178502    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy"
Oct 30 20:00:00.178505 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178507    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c50ac16c-f6e9-42f5-8e35-2cab4009e3bb" containerName="istio-proxy"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178523    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178528    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178536    5232 cpu_manager.go:414] "containerMap: removing container" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178541    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178549    5232 cpu_manager.go:414] "containerMap: removing container" podUID="198f0b29-1953-4a6b-88ee-638cd1f6e114" containerName="maas-api"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178554    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="198f0b29-1953-4a6b-88ee-638cd1f6e114" containerName="maas-api"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178564    5232 cpu_manager.go:414] "containerMap: removing container" podUID="827c3421-c6e8-404f-84b7-69cf1bd1088d" containerName="payload-processing"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178569    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="827c3421-c6e8-404f-84b7-69cf1bd1088d" containerName="payload-processing"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178577    5232 cpu_manager.go:414] "containerMap: removing container" podUID="7d447874-7803-46a2-bcbe-e8a9bffb33d4" containerName="maas-api"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178581    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d447874-7803-46a2-bcbe-e8a9bffb33d4" containerName="maas-api"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178672    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178680    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="56f9b6fd-b4e9-495c-bafe-71522b3d3469" containerName="payload-processing"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178688    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c0b309f-e728-4e7d-8ca8-1b3bd8e8a7ab" containerName="istio-proxy"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178694    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9b68dad-e4e3-4892-924a-2cb5c605fa18" containerName="main"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178700    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="198f0b29-1953-4a6b-88ee-638cd1f6e114" containerName="maas-api"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178707    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="827c3421-c6e8-404f-84b7-69cf1bd1088d" containerName="payload-processing"
Oct 30 20:00:00.178711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.178712    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d447874-7803-46a2-bcbe-e8a9bffb33d4" containerName="maas-api"
Oct 30 20:00:00.181528 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.181510    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:00.187246 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.187223    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"]
Oct 30 20:00:00.187674 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.187661    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="e69a2eb3-19ad-417f-854e-49b0c11167a6" containerName="istio-proxy"
Oct 30 20:00:00.190598 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.190579    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:00.211694 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.211670    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skflr\" (UniqueName: \"kubernetes.io/projected/54fbc7ed-96af-4797-a754-2e64b85ec7a3-kube-api-access-skflr\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48\" (UID: \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:00.211792 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.211712    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54fbc7ed-96af-4797-a754-2e64b85ec7a3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48\" (UID: \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:00.211881 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.211849    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks\" (UID: \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:00.211938 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.211904    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq9lh\" (UniqueName: \"kubernetes.io/projected/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-kube-api-access-xq9lh\") pod \"maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks\" (UID: \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:00.231911 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.231886    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 20:00:00.285406 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.285374    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"]
Oct 30 20:00:00.313123 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.313087    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skflr\" (UniqueName: \"kubernetes.io/projected/54fbc7ed-96af-4797-a754-2e64b85ec7a3-kube-api-access-skflr\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48\" (UID: \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:00.313337 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.313138    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54fbc7ed-96af-4797-a754-2e64b85ec7a3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48\" (UID: \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:00.313337 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.313204    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks\" (UID: \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:00.313337 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.313245    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq9lh\" (UniqueName: \"kubernetes.io/projected/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-kube-api-access-xq9lh\") pod \"maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks\" (UID: \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:00.313870 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.313847    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks\" (UID: \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:00.313925 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.313901    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54fbc7ed-96af-4797-a754-2e64b85ec7a3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48\" (UID: \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:00.321181 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.321158    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"]
Oct 30 20:00:00.372661 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.372632    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skflr\" (UniqueName: \"kubernetes.io/projected/54fbc7ed-96af-4797-a754-2e64b85ec7a3-kube-api-access-skflr\") pod \"maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48\" (UID: \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:00.372853 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.372738    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq9lh\" (UniqueName: \"kubernetes.io/projected/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-kube-api-access-xq9lh\") pod \"maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks\" (UID: \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:00.494082 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.494002    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:00.502187 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.502159    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:00.868582 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.868546    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"]
Oct 30 20:00:00.873391 ip-10-0-143-248 kubenswrapper[5232]: W1030 20:00:00.873364    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54fbc7ed_96af_4797_a754_2e64b85ec7a3.slice/crio-5105e78c22ce538996f885ad6837afb1c2fb4ee943d07373a2cf0ef743aae698 WatchSource:0}: Error finding container 5105e78c22ce538996f885ad6837afb1c2fb4ee943d07373a2cf0ef743aae698: Status 404 returned error can't find the container with id 5105e78c22ce538996f885ad6837afb1c2fb4ee943d07373a2cf0ef743aae698
Oct 30 20:00:00.919937 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:00.919910    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"]
Oct 30 20:00:00.924099 ip-10-0-143-248 kubenswrapper[5232]: W1030 20:00:00.924073    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d07fdd_e6f4_40e0_bea5_3e05fa5d2a32.slice/crio-4935745925cd33eaf01cfb995bf6b701ab6c8e1c06e0cd1bf01ef2fc03fe76a8 WatchSource:0}: Error finding container 4935745925cd33eaf01cfb995bf6b701ab6c8e1c06e0cd1bf01ef2fc03fe76a8: Status 404 returned error can't find the container with id 4935745925cd33eaf01cfb995bf6b701ab6c8e1c06e0cd1bf01ef2fc03fe76a8
Oct 30 20:00:01.046331 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:01.046296    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" event={"ID":"54fbc7ed-96af-4797-a754-2e64b85ec7a3","Type":"ContainerStarted","Data":"5105e78c22ce538996f885ad6837afb1c2fb4ee943d07373a2cf0ef743aae698"}
Oct 30 20:00:01.047507 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:01.047478    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" event={"ID":"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32","Type":"ContainerStarted","Data":"4935745925cd33eaf01cfb995bf6b701ab6c8e1c06e0cd1bf01ef2fc03fe76a8"}
Oct 30 20:00:02.055049 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.054964    5232 generic.go:356] "Generic (PLEG): container finished" podID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerID="d729e34d8e3a0ce2e8ed16473ed6eb94a13e68caf10afbb4ee9d7f0ecadf0a15" exitCode=60
Oct 30 20:00:02.055437 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.055051    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" event={"ID":"54fbc7ed-96af-4797-a754-2e64b85ec7a3","Type":"ContainerDied","Data":"d729e34d8e3a0ce2e8ed16473ed6eb94a13e68caf10afbb4ee9d7f0ecadf0a15"}
Oct 30 20:00:02.055437 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.055282    5232 scope.go:122] "RemoveContainer" containerID="d729e34d8e3a0ce2e8ed16473ed6eb94a13e68caf10afbb4ee9d7f0ecadf0a15"
Oct 30 20:00:02.057148 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.057127    5232 generic.go:356] "Generic (PLEG): container finished" podID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerID="8cda5131bab228fc86bf7db205334d27fb4f5e7fe2a3788f13a8c80a7d4aee5b" exitCode=60
Oct 30 20:00:02.057264 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.057229    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" event={"ID":"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32","Type":"ContainerDied","Data":"8cda5131bab228fc86bf7db205334d27fb4f5e7fe2a3788f13a8c80a7d4aee5b"}
Oct 30 20:00:02.057495 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.057477    5232 scope.go:122] "RemoveContainer" containerID="8cda5131bab228fc86bf7db205334d27fb4f5e7fe2a3788f13a8c80a7d4aee5b"
Oct 30 20:00:02.488885 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.488855    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 20:00:02.537728 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.537688    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-certs\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.537961 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.537740    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-credential-socket\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.537961 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.537790    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ed138b6-03f5-460e-bd64-067bba10054f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed138b6-03f5-460e-bd64-067bba10054f-istiod-ca-cert\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.537961 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.537859    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-data\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.537961 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.537899    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-socket\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.538202 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538042    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-credential-socket" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:02.538202 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538068    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-certs" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:02.538202 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538098    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-istio-token\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-istio-token\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.538202 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538134    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ed138b6-03f5-460e-bd64-067bba10054f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed138b6-03f5-460e-bd64-067bba10054f-istio-podinfo\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.538435 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538249    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-envoy\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.538435 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538272    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-socket" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:02.538435 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538280    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-data" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:02.538435 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538311    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-kube-api-access-d7wpk\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-kube-api-access-d7wpk\") pod \"0ed138b6-03f5-460e-bd64-067bba10054f\" (UID: \"0ed138b6-03f5-460e-bd64-067bba10054f\") "
Oct 30 20:00:02.538435 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538371    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ed138b6-03f5-460e-bd64-067bba10054f-istiod-ca-cert" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:02.538727 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538668    5232 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.538727 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538688    5232 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-credential-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.538727 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538703    5232 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed138b6-03f5-460e-bd64-067bba10054f-istiod-ca-cert\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.538727 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538719    5232 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-data\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.539021 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.538732    5232 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-workload-socket\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.540483 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.540458    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-envoy" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:02.540655 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.540637    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-istio-token" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:02.540727 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.540705    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ed138b6-03f5-460e-bd64-067bba10054f-istio-podinfo" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:00:02.540967 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.540952    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-kube-api-access-d7wpk" pod "0ed138b6-03f5-460e-bd64-067bba10054f" (UID: "0ed138b6-03f5-460e-bd64-067bba10054f"). InnerVolumeSpecName "kube-api-access-d7wpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:02.639880 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.639768    5232 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed138b6-03f5-460e-bd64-067bba10054f-istio-podinfo\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.639880 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.639840    5232 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed138b6-03f5-460e-bd64-067bba10054f-istio-envoy\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.639880 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.639862    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7wpk\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-kube-api-access-d7wpk\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.640204 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.639888    5232 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed138b6-03f5-460e-bd64-067bba10054f-istio-token\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:02.654280 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.654247    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"]
Oct 30 20:00:02.665428 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:02.665394    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"]
Oct 30 20:00:03.065243 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.065210    5232 generic.go:356] "Generic (PLEG): container finished" podID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerID="602315e4665b5e5a4909db8b3e93d8d3ae33c0cc776b40b18d4730b0f5afc7b3" exitCode=60
Oct 30 20:00:03.065784 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.065304    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" event={"ID":"54fbc7ed-96af-4797-a754-2e64b85ec7a3","Type":"ContainerDied","Data":"602315e4665b5e5a4909db8b3e93d8d3ae33c0cc776b40b18d4730b0f5afc7b3"}
Oct 30 20:00:03.065784 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.065376    5232 scope.go:122] "RemoveContainer" containerID="d729e34d8e3a0ce2e8ed16473ed6eb94a13e68caf10afbb4ee9d7f0ecadf0a15"
Oct 30 20:00:03.065784 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.065640    5232 scope.go:122] "RemoveContainer" containerID="602315e4665b5e5a4909db8b3e93d8d3ae33c0cc776b40b18d4730b0f5afc7b3"
Oct 30 20:00:03.065990 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:03.065938    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48_odh-ai-gateway-infra(54fbc7ed-96af-4797-a754-2e64b85ec7a3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3"
Oct 30 20:00:03.067272 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.067250    5232 generic.go:356] "Generic (PLEG): container finished" podID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerID="cca4394c954143bf42c8d8b841f821d05c003c3d4f439cccbdfa98463a6484e4" exitCode=60
Oct 30 20:00:03.067363 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.067333    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" event={"ID":"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32","Type":"ContainerDied","Data":"cca4394c954143bf42c8d8b841f821d05c003c3d4f439cccbdfa98463a6484e4"}
Oct 30 20:00:03.067578 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.067561    5232 scope.go:122] "RemoveContainer" containerID="cca4394c954143bf42c8d8b841f821d05c003c3d4f439cccbdfa98463a6484e4"
Oct 30 20:00:03.067826 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:03.067791    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks_odh-ai-gateway-infra(c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32"
Oct 30 20:00:03.069000 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.068983    5232 generic.go:356] "Generic (PLEG): container finished" podID="0ed138b6-03f5-460e-bd64-067bba10054f" containerID="a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82" exitCode=0
Oct 30 20:00:03.069126 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.069036    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"
Oct 30 20:00:03.069126 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.069066    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" event={"ID":"0ed138b6-03f5-460e-bd64-067bba10054f","Type":"ContainerDied","Data":"a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82"}
Oct 30 20:00:03.069126 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.069119    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7" event={"ID":"0ed138b6-03f5-460e-bd64-067bba10054f","Type":"ContainerDied","Data":"67ade79c5f4962f8b138aa7950b28da145a81bf68d129dc61deacf54be681562"}
Oct 30 20:00:03.077428 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.077393    5232 scope.go:122] "RemoveContainer" containerID="8cda5131bab228fc86bf7db205334d27fb4f5e7fe2a3788f13a8c80a7d4aee5b"
Oct 30 20:00:03.091294 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.091271    5232 scope.go:122] "RemoveContainer" containerID="a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82"
Oct 30 20:00:03.100762 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.100736    5232 scope.go:122] "RemoveContainer" containerID="8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b"
Oct 30 20:00:03.109529 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.109510    5232 scope.go:122] "RemoveContainer" containerID="a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82"
Oct 30 20:00:03.109843 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:03.109796    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82\": container with ID starting with a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82 not found: ID does not exist" containerID="a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82"
Oct 30 20:00:03.109936 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.109852    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82"} err="failed to get container status \"a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82\": rpc error: code = NotFound desc = could not find container \"a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82\": container with ID starting with a7e7adbd0a688c7ea11a33f9852e5d0acd61593fa8da3bb1f1b836029c147e82 not found: ID does not exist"
Oct 30 20:00:03.109936 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.109873    5232 scope.go:122] "RemoveContainer" containerID="8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b"
Oct 30 20:00:03.110170 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:03.110149    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b\": container with ID starting with 8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b not found: ID does not exist" containerID="8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b"
Oct 30 20:00:03.110218 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.110177    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b"} err="failed to get container status \"8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b\": rpc error: code = NotFound desc = could not find container \"8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b\": container with ID starting with 8555d5311ffb84f4131252b8e6e32f30d85f02cd2c614bcdd2f289175d247e1b not found: ID does not exist"
Oct 30 20:00:03.285115 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.285028    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"]
Oct 30 20:00:03.289694 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.289665    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-c01685-openshift-default-7688cdd84c-nklc7"]
Oct 30 20:00:03.539702 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:03.539625    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" path="/var/lib/kubelet/pods/0ed138b6-03f5-460e-bd64-067bba10054f/volumes"
Oct 30 20:00:04.077085 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:04.077052    5232 scope.go:122] "RemoveContainer" containerID="602315e4665b5e5a4909db8b3e93d8d3ae33c0cc776b40b18d4730b0f5afc7b3"
Oct 30 20:00:04.077574 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:04.077371    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48_odh-ai-gateway-infra(54fbc7ed-96af-4797-a754-2e64b85ec7a3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3"
Oct 30 20:00:04.078662 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:04.078630    5232 scope.go:122] "RemoveContainer" containerID="cca4394c954143bf42c8d8b841f821d05c003c3d4f439cccbdfa98463a6484e4"
Oct 30 20:00:04.078939 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:04.078912    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks_odh-ai-gateway-infra(c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32"
Oct 30 20:00:09.638183 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:09.638146    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"]
Oct 30 20:00:14.639374 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:14.639326    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87" podUID="b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" containerName="payload-processing" containerID="cri-o://1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081" gracePeriod=25
Oct 30 20:00:15.533839 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:15.533785    5232 scope.go:122] "RemoveContainer" containerID="602315e4665b5e5a4909db8b3e93d8d3ae33c0cc776b40b18d4730b0f5afc7b3"
Oct 30 20:00:16.151543 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:16.151455    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" event={"ID":"54fbc7ed-96af-4797-a754-2e64b85ec7a3","Type":"ContainerStarted","Data":"d9122935a85f0775d260097fa598a90919a8ed6351b72d5a2fd643418114def5"}
Oct 30 20:00:16.188108 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:16.188063    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:00:16.265381 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:16.265331    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" podStartSLOduration=16.265317286 podStartE2EDuration="16.265317286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:16.263195409 +0000 UTC m=+4408.847678406" watchObservedRunningTime="2026-10-30 20:00:16.265317286 +0000 UTC m=+4408.849800282"
Oct 30 20:00:17.160163 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.160127    5232 generic.go:356] "Generic (PLEG): container finished" podID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerID="d9122935a85f0775d260097fa598a90919a8ed6351b72d5a2fd643418114def5" exitCode=60
Oct 30 20:00:17.160648 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.160200    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" event={"ID":"54fbc7ed-96af-4797-a754-2e64b85ec7a3","Type":"ContainerDied","Data":"d9122935a85f0775d260097fa598a90919a8ed6351b72d5a2fd643418114def5"}
Oct 30 20:00:17.160648 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.160277    5232 scope.go:122] "RemoveContainer" containerID="602315e4665b5e5a4909db8b3e93d8d3ae33c0cc776b40b18d4730b0f5afc7b3"
Oct 30 20:00:17.160648 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.160625    5232 scope.go:122] "RemoveContainer" containerID="d9122935a85f0775d260097fa598a90919a8ed6351b72d5a2fd643418114def5"
Oct 30 20:00:17.160905 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:17.160885    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48_odh-ai-gateway-infra(54fbc7ed-96af-4797-a754-2e64b85ec7a3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3"
Oct 30 20:00:17.286787 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.286741    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"]
Oct 30 20:00:17.537774 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.537688    5232 scope.go:122] "RemoveContainer" containerID="cca4394c954143bf42c8d8b841f821d05c003c3d4f439cccbdfa98463a6484e4"
Oct 30 20:00:17.675147 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.675102    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" containerID="cri-o://c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b" gracePeriod=45
Oct 30 20:00:17.686432 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.686388    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main" containerID="cri-o://2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c" gracePeriod=45
Oct 30 20:00:17.952955 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.952924    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 20:00:17.976505 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:17.976475    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 20:00:18.096236 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096202    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-model-cache\") pod \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") "
Oct 30 20:00:18.096450 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096278    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-dshm\") pod \"f56e5bbf-8007-4b3a-af62-006001102f1e\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") "
Oct 30 20:00:18.096450 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096309    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-dshm\") pod \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") "
Oct 30 20:00:18.096450 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096341    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-home\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-home\") pod \"f56e5bbf-8007-4b3a-af62-006001102f1e\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") "
Oct 30 20:00:18.096450 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096373    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-model-cache\") pod \"f56e5bbf-8007-4b3a-af62-006001102f1e\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") "
Oct 30 20:00:18.096450 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096437    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-tmp-dir\") pod \"f56e5bbf-8007-4b3a-af62-006001102f1e\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") "
Oct 30 20:00:18.096740 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096492    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f56e5bbf-8007-4b3a-af62-006001102f1e-kube-api-access-nvqm4\" (UniqueName: \"kubernetes.io/projected/f56e5bbf-8007-4b3a-af62-006001102f1e-kube-api-access-nvqm4\") pod \"f56e5bbf-8007-4b3a-af62-006001102f1e\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") "
Oct 30 20:00:18.096740 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096547    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tmp-dir\") pod \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") "
Oct 30 20:00:18.096740 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096618    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-kube-api-access-6sh8n\" (UniqueName: \"kubernetes.io/projected/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-kube-api-access-6sh8n\") pod \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") "
Oct 30 20:00:18.096740 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096661    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tls-certs\" (UniqueName: \"kubernetes.io/secret/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tls-certs\") pod \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") "
Oct 30 20:00:18.096740 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096696    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-home\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-home\") pod \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\" (UID: \"bbabad87-3446-4c59-94a0-ef9bd2cf31c0\") "
Oct 30 20:00:18.097027 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.096773    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f56e5bbf-8007-4b3a-af62-006001102f1e-tls-certs\" (UniqueName: \"kubernetes.io/secret/f56e5bbf-8007-4b3a-af62-006001102f1e-tls-certs\") pod \"f56e5bbf-8007-4b3a-af62-006001102f1e\" (UID: \"f56e5bbf-8007-4b3a-af62-006001102f1e\") "
Oct 30 20:00:18.097800 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.097757    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-tmp-dir" pod "f56e5bbf-8007-4b3a-af62-006001102f1e" (UID: "f56e5bbf-8007-4b3a-af62-006001102f1e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:18.098053 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.098009    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-home" pod "f56e5bbf-8007-4b3a-af62-006001102f1e" (UID: "f56e5bbf-8007-4b3a-af62-006001102f1e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:18.098213 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.098195    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-model-cache" pod "f56e5bbf-8007-4b3a-af62-006001102f1e" (UID: "f56e5bbf-8007-4b3a-af62-006001102f1e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:18.098316 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.098287    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-model-cache" pod "bbabad87-3446-4c59-94a0-ef9bd2cf31c0" (UID: "bbabad87-3446-4c59-94a0-ef9bd2cf31c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:18.098947 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.098889    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tmp-dir" pod "bbabad87-3446-4c59-94a0-ef9bd2cf31c0" (UID: "bbabad87-3446-4c59-94a0-ef9bd2cf31c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:18.099131 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.099092    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-home" pod "bbabad87-3446-4c59-94a0-ef9bd2cf31c0" (UID: "bbabad87-3446-4c59-94a0-ef9bd2cf31c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:18.100764 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.100740    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-dshm" pod "f56e5bbf-8007-4b3a-af62-006001102f1e" (UID: "f56e5bbf-8007-4b3a-af62-006001102f1e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:18.101003 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.100929    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f56e5bbf-8007-4b3a-af62-006001102f1e-tls-certs" pod "f56e5bbf-8007-4b3a-af62-006001102f1e" (UID: "f56e5bbf-8007-4b3a-af62-006001102f1e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:00:18.101244 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.101217    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tls-certs" pod "bbabad87-3446-4c59-94a0-ef9bd2cf31c0" (UID: "bbabad87-3446-4c59-94a0-ef9bd2cf31c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:00:18.102086 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.102060    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-kube-api-access-6sh8n" pod "bbabad87-3446-4c59-94a0-ef9bd2cf31c0" (UID: "bbabad87-3446-4c59-94a0-ef9bd2cf31c0"). InnerVolumeSpecName "kube-api-access-6sh8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:18.102407 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.102385    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56e5bbf-8007-4b3a-af62-006001102f1e-kube-api-access-nvqm4" pod "f56e5bbf-8007-4b3a-af62-006001102f1e" (UID: "f56e5bbf-8007-4b3a-af62-006001102f1e"). InnerVolumeSpecName "kube-api-access-nvqm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:18.102587 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.102568    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-dshm" pod "bbabad87-3446-4c59-94a0-ef9bd2cf31c0" (UID: "bbabad87-3446-4c59-94a0-ef9bd2cf31c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:18.171108 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.171071    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" event={"ID":"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32","Type":"ContainerStarted","Data":"b7212caa1ffe38ac38067484875ff33aa4da14a1ef4fa050f26505be8320ee11"}
Oct 30 20:00:18.172541 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.172512    5232 generic.go:356] "Generic (PLEG): container finished" podID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerID="c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b" exitCode=0
Oct 30 20:00:18.172688 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.172594    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"
Oct 30 20:00:18.172759 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.172600    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" event={"ID":"f56e5bbf-8007-4b3a-af62-006001102f1e","Type":"ContainerDied","Data":"c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b"}
Oct 30 20:00:18.172759 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.172743    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" event={"ID":"f56e5bbf-8007-4b3a-af62-006001102f1e","Type":"ContainerDied","Data":"db95e6800625c9dbacae228d300400c0bee37c074a9c2881008d1e1d5c6adede"}
Oct 30 20:00:18.172896 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.172771    5232 scope.go:122] "RemoveContainer" containerID="c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b"
Oct 30 20:00:18.176494 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.176467    5232 generic.go:356] "Generic (PLEG): container finished" podID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerID="2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c" exitCode=0
Oct 30 20:00:18.176667 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.176522    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"
Oct 30 20:00:18.176667 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.176567    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" event={"ID":"bbabad87-3446-4c59-94a0-ef9bd2cf31c0","Type":"ContainerDied","Data":"2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c"}
Oct 30 20:00:18.176667 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.176597    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm" event={"ID":"bbabad87-3446-4c59-94a0-ef9bd2cf31c0","Type":"ContainerDied","Data":"50ae493dbc39caa34319da88f0d95d6d3bc12531bd0cd4b3951e2181dd1d88d4"}
Oct 30 20:00:18.185243 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.185223    5232 scope.go:122] "RemoveContainer" containerID="c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b"
Oct 30 20:00:18.185613 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:18.185580    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b\": container with ID starting with c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b not found: ID does not exist" containerID="c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b"
Oct 30 20:00:18.185715 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.185626    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b"} err="failed to get container status \"c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b\": rpc error: code = NotFound desc = could not find container \"c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b\": container with ID starting with c4c5044ada85d9712d20272be7f1ec44a6b878ada2c73b2d72ab0883b05e4e8b not found: ID does not exist"
Oct 30 20:00:18.185715 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.185648    5232 scope.go:122] "RemoveContainer" containerID="2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c"
Oct 30 20:00:18.196054 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.196032    5232 scope.go:122] "RemoveContainer" containerID="2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c"
Oct 30 20:00:18.196459 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:18.196436    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c\": container with ID starting with 2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c not found: ID does not exist" containerID="2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c"
Oct 30 20:00:18.196538 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.196474    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c"} err="failed to get container status \"2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c\": rpc error: code = NotFound desc = could not find container \"2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c\": container with ID starting with 2a7b392a778ef492e68f7a3d3e83d6853d0148007b5bf1d5ff0b56f578bd465c not found: ID does not exist"
Oct 30 20:00:18.198464 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198432    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.198590 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198470    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sh8n\" (UniqueName: \"kubernetes.io/projected/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-kube-api-access-6sh8n\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.198590 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198487    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.198590 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198499    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.198590 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198515    5232 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f56e5bbf-8007-4b3a-af62-006001102f1e-tls-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.198590 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198527    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.198590 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198541    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.198590 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198554    5232 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbabad87-3446-4c59-94a0-ef9bd2cf31c0-dshm\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.199294 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198566    5232 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-home\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.199294 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198615    5232 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-model-cache\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.199294 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198627    5232 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f56e5bbf-8007-4b3a-af62-006001102f1e-tmp-dir\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.199294 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.198941    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvqm4\" (UniqueName: \"kubernetes.io/projected/f56e5bbf-8007-4b3a-af62-006001102f1e-kube-api-access-nvqm4\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.318611 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.318561    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" podStartSLOduration=18.31854691 podStartE2EDuration="18.31854691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:18.316916679 +0000 UTC m=+4410.901399678" watchObservedRunningTime="2026-10-30 20:00:18.31854691 +0000 UTC m=+4410.903029907"
Oct 30 20:00:18.327346 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.327316    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:18.401146 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.401056    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54fbc7ed-96af-4797-a754-2e64b85ec7a3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54fbc7ed-96af-4797-a754-2e64b85ec7a3-maas-api-service-ca\") pod \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\" (UID: \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\") "
Oct 30 20:00:18.401146 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.401118    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54fbc7ed-96af-4797-a754-2e64b85ec7a3-kube-api-access-skflr\" (UniqueName: \"kubernetes.io/projected/54fbc7ed-96af-4797-a754-2e64b85ec7a3-kube-api-access-skflr\") pod \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\" (UID: \"54fbc7ed-96af-4797-a754-2e64b85ec7a3\") "
Oct 30 20:00:18.401521 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.401493    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54fbc7ed-96af-4797-a754-2e64b85ec7a3-maas-api-service-ca" pod "54fbc7ed-96af-4797-a754-2e64b85ec7a3" (UID: "54fbc7ed-96af-4797-a754-2e64b85ec7a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:18.403265 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.403246    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54fbc7ed-96af-4797-a754-2e64b85ec7a3-kube-api-access-skflr" pod "54fbc7ed-96af-4797-a754-2e64b85ec7a3" (UID: "54fbc7ed-96af-4797-a754-2e64b85ec7a3"). InnerVolumeSpecName "kube-api-access-skflr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:18.502539 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.502497    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54fbc7ed-96af-4797-a754-2e64b85ec7a3-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.502539 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.502528    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skflr\" (UniqueName: \"kubernetes.io/projected/54fbc7ed-96af-4797-a754-2e64b85ec7a3-kube-api-access-skflr\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:18.539713 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.539674    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"]
Oct 30 20:00:18.544656 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.544625    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-330025/premium-w2-330025-kserve-79956d98fc-lgwbm"]
Oct 30 20:00:18.652732 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.652641    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"]
Oct 30 20:00:18.675771 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.675738    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg"]
Oct 30 20:00:18.943573 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:18.943483    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-330025/distinct-w2-330025-kserve-55d67597b4-z4vcg" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:00:19.186349 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.186316    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"
Oct 30 20:00:19.186861 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.186317    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48" event={"ID":"54fbc7ed-96af-4797-a754-2e64b85ec7a3","Type":"ContainerDied","Data":"5105e78c22ce538996f885ad6837afb1c2fb4ee943d07373a2cf0ef743aae698"}
Oct 30 20:00:19.186861 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.186465    5232 scope.go:122] "RemoveContainer" containerID="d9122935a85f0775d260097fa598a90919a8ed6351b72d5a2fd643418114def5"
Oct 30 20:00:19.188412 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.188385    5232 generic.go:356] "Generic (PLEG): container finished" podID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerID="b7212caa1ffe38ac38067484875ff33aa4da14a1ef4fa050f26505be8320ee11" exitCode=60
Oct 30 20:00:19.188517 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.188419    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" event={"ID":"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32","Type":"ContainerDied","Data":"b7212caa1ffe38ac38067484875ff33aa4da14a1ef4fa050f26505be8320ee11"}
Oct 30 20:00:19.188915 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.188896    5232 scope.go:122] "RemoveContainer" containerID="b7212caa1ffe38ac38067484875ff33aa4da14a1ef4fa050f26505be8320ee11"
Oct 30 20:00:19.189220 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:19.189196    5232 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks_odh-ai-gateway-infra(c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32"
Oct 30 20:00:19.195886 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.195854    5232 scope.go:122] "RemoveContainer" containerID="cca4394c954143bf42c8d8b841f821d05c003c3d4f439cccbdfa98463a6484e4"
Oct 30 20:00:19.326168 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.326128    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"]
Oct 30 20:00:19.480196 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.480098    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"]
Oct 30 20:00:19.483236 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.483209    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-1323a7-29889840-hpt48"]
Oct 30 20:00:19.539707 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.539670    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" path="/var/lib/kubelet/pods/54fbc7ed-96af-4797-a754-2e64b85ec7a3/volumes"
Oct 30 20:00:19.540254 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.540231    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" path="/var/lib/kubelet/pods/bbabad87-3446-4c59-94a0-ef9bd2cf31c0/volumes"
Oct 30 20:00:19.540662 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:19.540643    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" path="/var/lib/kubelet/pods/f56e5bbf-8007-4b3a-af62-006001102f1e/volumes"
Oct 30 20:00:20.330738 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:20.330711    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:20.420043 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:20.420007    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-maas-api-service-ca\") pod \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\" (UID: \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\") "
Oct 30 20:00:20.420239 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:20.420098    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-kube-api-access-xq9lh\" (UniqueName: \"kubernetes.io/projected/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-kube-api-access-xq9lh\") pod \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\" (UID: \"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32\") "
Oct 30 20:00:20.420478 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:20.420451    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-maas-api-service-ca" pod "c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" (UID: "c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:20.422232 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:20.422202    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-kube-api-access-xq9lh" pod "c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" (UID: "c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32"). InnerVolumeSpecName "kube-api-access-xq9lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:20.521741 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:20.521652    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-maas-api-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:20.521741 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:20.521686    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq9lh\" (UniqueName: \"kubernetes.io/projected/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32-kube-api-access-xq9lh\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:21.211824 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:21.211772    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks" event={"ID":"c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32","Type":"ContainerDied","Data":"4935745925cd33eaf01cfb995bf6b701ab6c8e1c06e0cd1bf01ef2fc03fe76a8"}
Oct 30 20:00:21.211824 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:21.211803    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"
Oct 30 20:00:21.212084 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:21.211907    5232 scope.go:122] "RemoveContainer" containerID="b7212caa1ffe38ac38067484875ff33aa4da14a1ef4fa050f26505be8320ee11"
Oct 30 20:00:21.370656 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:21.370618    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"]
Oct 30 20:00:21.418003 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:21.417971    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-d50b07-29889840-56hks"]
Oct 30 20:00:21.539092 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:21.539011    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" path="/var/lib/kubelet/pods/c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32/volumes"
Oct 30 20:00:22.403228 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:22.403190    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87" podUID="b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:00:32.402857 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:32.402758    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87" podUID="b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:00:34.889988 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:34.889958    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 20:00:34.956706 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:34.956625    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-plugins-config-volume\") pod \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\" (UID: \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\") "
Oct 30 20:00:34.956706 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:34.956662    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-kube-api-access-bq49d\" (UniqueName: \"kubernetes.io/projected/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-kube-api-access-bq49d\") pod \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\" (UID: \"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3\") "
Oct 30 20:00:34.957049 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:34.957025    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-plugins-config-volume" pod "b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" (UID: "b3e7fe22-59ed-43d3-8fc8-36b88aa829a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:34.958746 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:34.958725    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-kube-api-access-bq49d" pod "b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" (UID: "b3e7fe22-59ed-43d3-8fc8-36b88aa829a3"). InnerVolumeSpecName "kube-api-access-bq49d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:35.057442 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.057403    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:35.057442 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.057439    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq49d\" (UniqueName: \"kubernetes.io/projected/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3-kube-api-access-bq49d\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:35.290054 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.289973    5232 generic.go:356] "Generic (PLEG): container finished" podID="b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" containerID="1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081" exitCode=0
Oct 30 20:00:35.290054 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.290020    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"
Oct 30 20:00:35.290250 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.290064    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87" event={"ID":"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3","Type":"ContainerDied","Data":"1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081"}
Oct 30 20:00:35.290250 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.290116    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87" event={"ID":"b3e7fe22-59ed-43d3-8fc8-36b88aa829a3","Type":"ContainerDied","Data":"f318b41445ad37e12234d8e34fb9877d9101f3ec8177ebdc350133d13451b573"}
Oct 30 20:00:35.290250 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.290140    5232 scope.go:122] "RemoveContainer" containerID="1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081"
Oct 30 20:00:35.299215 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.299198    5232 scope.go:122] "RemoveContainer" containerID="1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081"
Oct 30 20:00:35.299468 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:35.299451    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081\": container with ID starting with 1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081 not found: ID does not exist" containerID="1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081"
Oct 30 20:00:35.299506 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.299475    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081"} err="failed to get container status \"1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081\": rpc error: code = NotFound desc = could not find container \"1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081\": container with ID starting with 1ff60c86d7c7ea13c7480df11f7fb68cdeee8e0c2053adb8271628d586bfe081 not found: ID does not exist"
Oct 30 20:00:35.455601 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.455563    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"]
Oct 30 20:00:35.461821 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.461460    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-330025-bbdbdf888-nck87"]
Oct 30 20:00:35.538700 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:35.538661    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" path="/var/lib/kubelet/pods/b3e7fe22-59ed-43d3-8fc8-36b88aa829a3/volumes"
Oct 30 20:00:46.187618 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:00:46.187582    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:00:49.334679 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:00:49.334643    5232 scope.go:122] "RemoveContainer" containerID="3daf776f6b0a58a65e458e9640ad02bef3d91ab785be57b052f6872c4ae30eee"
Oct 30 20:01:16.187807 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:01:16.187762    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:01:43.936166 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:43.936130    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"]
Oct 30 20:01:43.936808 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:43.936372    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57" podUID="d3422719-5018-4802-9df7-13a780733fbb" containerName="maas-api" containerID="cri-o://78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b" gracePeriod=30
Oct 30 20:01:44.191663 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.191600    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 20:01:44.335978 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.335943    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-metrics-certs\") pod \"d3422719-5018-4802-9df7-13a780733fbb\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") "
Oct 30 20:01:44.336172 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.336001    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3422719-5018-4802-9df7-13a780733fbb-kube-api-access-prg4x\" (UniqueName: \"kubernetes.io/projected/d3422719-5018-4802-9df7-13a780733fbb-kube-api-access-prg4x\") pod \"d3422719-5018-4802-9df7-13a780733fbb\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") "
Oct 30 20:01:44.336172 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.336045    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls\") pod \"d3422719-5018-4802-9df7-13a780733fbb\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") "
Oct 30 20:01:44.336172 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.336082    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3422719-5018-4802-9df7-13a780733fbb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3422719-5018-4802-9df7-13a780733fbb-openshift-service-ca\") pod \"d3422719-5018-4802-9df7-13a780733fbb\" (UID: \"d3422719-5018-4802-9df7-13a780733fbb\") "
Oct 30 20:01:44.336569 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.336537    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3422719-5018-4802-9df7-13a780733fbb-openshift-service-ca" pod "d3422719-5018-4802-9df7-13a780733fbb" (UID: "d3422719-5018-4802-9df7-13a780733fbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:01:44.338131 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.338103    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-metrics-certs" pod "d3422719-5018-4802-9df7-13a780733fbb" (UID: "d3422719-5018-4802-9df7-13a780733fbb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:01:44.338389 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.338172    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3422719-5018-4802-9df7-13a780733fbb-kube-api-access-prg4x" pod "d3422719-5018-4802-9df7-13a780733fbb" (UID: "d3422719-5018-4802-9df7-13a780733fbb"). InnerVolumeSpecName "kube-api-access-prg4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:01:44.338577 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.338552    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls" pod "d3422719-5018-4802-9df7-13a780733fbb" (UID: "d3422719-5018-4802-9df7-13a780733fbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:01:44.436920 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.436886    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prg4x\" (UniqueName: \"kubernetes.io/projected/d3422719-5018-4802-9df7-13a780733fbb-kube-api-access-prg4x\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:01:44.436920 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.436917    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:01:44.436920 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.436927    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3422719-5018-4802-9df7-13a780733fbb-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:01:44.437170 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.436936    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3422719-5018-4802-9df7-13a780733fbb-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:01:44.664197 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.664161    5232 generic.go:356] "Generic (PLEG): container finished" podID="d3422719-5018-4802-9df7-13a780733fbb" containerID="78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b" exitCode=0
Oct 30 20:01:44.664400 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.664211    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"
Oct 30 20:01:44.664400 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.664247    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57" event={"ID":"d3422719-5018-4802-9df7-13a780733fbb","Type":"ContainerDied","Data":"78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b"}
Oct 30 20:01:44.664400 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.664310    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57" event={"ID":"d3422719-5018-4802-9df7-13a780733fbb","Type":"ContainerDied","Data":"28ea26a4ccd51743180add149ae222631fa086e62abf924565cb3827d29b92fe"}
Oct 30 20:01:44.664400 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.664341    5232 scope.go:122] "RemoveContainer" containerID="78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b"
Oct 30 20:01:44.673625 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.673607    5232 scope.go:122] "RemoveContainer" containerID="78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b"
Oct 30 20:01:44.673923 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:01:44.673902    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b\": container with ID starting with 78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b not found: ID does not exist" containerID="78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b"
Oct 30 20:01:44.674014 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.673934    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b"} err="failed to get container status \"78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b\": rpc error: code = NotFound desc = could not find container \"78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b\": container with ID starting with 78dbcc4a3a6b519dbb2140dd82b003fa517af7615e2d3c06c95af2f6a89cd20b not found: ID does not exist"
Oct 30 20:01:44.823686 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.823650    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"]
Oct 30 20:01:44.827164 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:44.827127    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d50b07-db64d5b97-tnj57"]
Oct 30 20:01:45.538733 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:45.538700    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3422719-5018-4802-9df7-13a780733fbb" path="/var/lib/kubelet/pods/d3422719-5018-4802-9df7-13a780733fbb/volumes"
Oct 30 20:01:46.186399 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:01:46.186362    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:01:48.254510 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:48.254478    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 20:01:48.258868 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:48.258848    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 20:01:48.283761 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:48.283741    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gnqpt_0b8e28d4-3c32-4801-8fa8-4a19a59fdb14/extract/0.log"
Oct 30 20:01:48.287653 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:01:48.287637    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 20:02:13.347053 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.347015    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"]
Oct 30 20:02:13.347548 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.347219    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d" podUID="23f11210-07cb-4964-a8e6-5364f05adff8" containerName="maas-api" containerID="cri-o://b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124" gracePeriod=30
Oct 30 20:02:13.596253 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.596224    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 20:02:13.710384 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.710345    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23f11210-07cb-4964-a8e6-5364f05adff8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f11210-07cb-4964-a8e6-5364f05adff8-openshift-service-ca\") pod \"23f11210-07cb-4964-a8e6-5364f05adff8\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") "
Oct 30 20:02:13.710563 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.710399    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-metrics-certs\") pod \"23f11210-07cb-4964-a8e6-5364f05adff8\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") "
Oct 30 20:02:13.710563 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.710436    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23f11210-07cb-4964-a8e6-5364f05adff8-kube-api-access-njspz\" (UniqueName: \"kubernetes.io/projected/23f11210-07cb-4964-a8e6-5364f05adff8-kube-api-access-njspz\") pod \"23f11210-07cb-4964-a8e6-5364f05adff8\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") "
Oct 30 20:02:13.710563 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.710471    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls\") pod \"23f11210-07cb-4964-a8e6-5364f05adff8\" (UID: \"23f11210-07cb-4964-a8e6-5364f05adff8\") "
Oct 30 20:02:13.710790 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.710762    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23f11210-07cb-4964-a8e6-5364f05adff8-openshift-service-ca" pod "23f11210-07cb-4964-a8e6-5364f05adff8" (UID: "23f11210-07cb-4964-a8e6-5364f05adff8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:13.712737 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.712709    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls" pod "23f11210-07cb-4964-a8e6-5364f05adff8" (UID: "23f11210-07cb-4964-a8e6-5364f05adff8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:13.712869 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.712761    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23f11210-07cb-4964-a8e6-5364f05adff8-kube-api-access-njspz" pod "23f11210-07cb-4964-a8e6-5364f05adff8" (UID: "23f11210-07cb-4964-a8e6-5364f05adff8"). InnerVolumeSpecName "kube-api-access-njspz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:13.712869 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.712790    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-metrics-certs" pod "23f11210-07cb-4964-a8e6-5364f05adff8" (UID: "23f11210-07cb-4964-a8e6-5364f05adff8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:13.812225 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.812135    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-maas-api-tls\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:13.812225 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.812168    5232 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23f11210-07cb-4964-a8e6-5364f05adff8-openshift-service-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:13.812225 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.812181    5232 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23f11210-07cb-4964-a8e6-5364f05adff8-metrics-certs\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:13.812225 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.812195    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njspz\" (UniqueName: \"kubernetes.io/projected/23f11210-07cb-4964-a8e6-5364f05adff8-kube-api-access-njspz\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:13.824652 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.824619    5232 generic.go:356] "Generic (PLEG): container finished" podID="23f11210-07cb-4964-a8e6-5364f05adff8" containerID="b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124" exitCode=0
Oct 30 20:02:13.824846 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.824671    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"
Oct 30 20:02:13.824846 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.824691    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d" event={"ID":"23f11210-07cb-4964-a8e6-5364f05adff8","Type":"ContainerDied","Data":"b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124"}
Oct 30 20:02:13.824846 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.824733    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d" event={"ID":"23f11210-07cb-4964-a8e6-5364f05adff8","Type":"ContainerDied","Data":"c9dd24c8f44096c7ee32d89e3b1ebfa6062f9e0d165d239a7273ab162d36878a"}
Oct 30 20:02:13.824846 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.824756    5232 scope.go:122] "RemoveContainer" containerID="b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124"
Oct 30 20:02:13.835128 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.835107    5232 scope.go:122] "RemoveContainer" containerID="b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124"
Oct 30 20:02:13.835399 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:02:13.835376    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124\": container with ID starting with b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124 not found: ID does not exist" containerID="b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124"
Oct 30 20:02:13.835467 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.835411    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124"} err="failed to get container status \"b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124\": rpc error: code = NotFound desc = could not find container \"b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124\": container with ID starting with b0657c6d455ebb80fcd96cbe11cfe03b3b7f40706d02659b109f38d2ba52c124 not found: ID does not exist"
Oct 30 20:02:13.988431 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.988397    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"]
Oct 30 20:02:13.993590 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:13.993562    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f77d05-5d8d8675df-tgh8d"]
Oct 30 20:02:15.539347 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:15.539313    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23f11210-07cb-4964-a8e6-5364f05adff8" path="/var/lib/kubelet/pods/23f11210-07cb-4964-a8e6-5364f05adff8/volumes"
Oct 30 20:02:16.187287 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:02:16.187177    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:02:24.231031 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.230991    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"]
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231439    5232 cpu_manager.go:414] "containerMap: removing container" podUID="23f11210-07cb-4964-a8e6-5364f05adff8" containerName="maas-api"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231457    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23f11210-07cb-4964-a8e6-5364f05adff8" containerName="maas-api"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231473    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231478    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231486    5232 cpu_manager.go:414] "containerMap: removing container" podUID="b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" containerName="payload-processing"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231491    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" containerName="payload-processing"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231501    5232 cpu_manager.go:414] "containerMap: removing container" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231507    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231516    5232 cpu_manager.go:414] "containerMap: removing container" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231521    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231527    5232 cpu_manager.go:414] "containerMap: removing container" podUID="d3422719-5018-4802-9df7-13a780733fbb" containerName="maas-api"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231533    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3422719-5018-4802-9df7-13a780733fbb" containerName="maas-api"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231542    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231546    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231554    5232 cpu_manager.go:414] "containerMap: removing container" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main"
Oct 30 20:02:24.231550 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231559    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231565    5232 cpu_manager.go:414] "containerMap: removing container" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231570    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231583    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231662    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231675    5232 cpu_manager.go:414] "containerMap: removing container" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231691    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231703    5232 cpu_manager.go:414] "containerMap: removing container" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231712    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231852    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="23f11210-07cb-4964-a8e6-5364f05adff8" containerName="maas-api"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231938    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231951    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3e7fe22-59ed-43d3-8fc8-36b88aa829a3" containerName="payload-processing"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231962    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ed138b6-03f5-460e-bd64-067bba10054f" containerName="istio-proxy"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231972    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbabad87-3446-4c59-94a0-ef9bd2cf31c0" containerName="main"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231980    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3422719-5018-4802-9df7-13a780733fbb" containerName="maas-api"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.231997    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.232008    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="f56e5bbf-8007-4b3a-af62-006001102f1e" containerName="main"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.232017    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.232029    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.232043    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.232050    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.232300 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.232058    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="54fbc7ed-96af-4797-a754-2e64b85ec7a3" containerName="cleanup"
Oct 30 20:02:24.237226 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.237198    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.310859 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.310803    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr2jq\" (UniqueName: \"kubernetes.io/projected/90a8fd79-7dd1-4836-8a50-cecd856be150-kube-api-access-hr2jq\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.311040 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.310968    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.311097 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.311062    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.334697 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.334662    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:02:24.334697 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.334680    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 20:02:24.336910 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.336888    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"]
Oct 30 20:02:24.411896 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.411859    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.412091 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.411917    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.412091 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.411991    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr2jq\" (UniqueName: \"kubernetes.io/projected/90a8fd79-7dd1-4836-8a50-cecd856be150-kube-api-access-hr2jq\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.420293 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.413047    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.420293 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.415291    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.471460 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.471421    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr2jq\" (UniqueName: \"kubernetes.io/projected/90a8fd79-7dd1-4836-8a50-cecd856be150-kube-api-access-hr2jq\") pod \"maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.550861 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.550758    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:24.923041 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:24.923006    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"]
Oct 30 20:02:24.928848 ip-10-0-143-248 kubenswrapper[5232]: W1030 20:02:24.928804    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90a8fd79_7dd1_4836_8a50_cecd856be150.slice/crio-b2a4f907d6b7a0ad45082ad8da6ecf063df5fe1ba92e741ac21c12ed832028a0 WatchSource:0}: Error finding container b2a4f907d6b7a0ad45082ad8da6ecf063df5fe1ba92e741ac21c12ed832028a0: Status 404 returned error can't find the container with id b2a4f907d6b7a0ad45082ad8da6ecf063df5fe1ba92e741ac21c12ed832028a0
Oct 30 20:02:25.892560 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:25.892520    5232 generic.go:356] "Generic (PLEG): container finished" podID="90a8fd79-7dd1-4836-8a50-cecd856be150" containerID="db1bce326d5437f7ead489c2e56becef3aa8c3c197f3754a3e611ae4b8e0d4a8" exitCode=0
Oct 30 20:02:25.892999 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:25.892612    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv" event={"ID":"90a8fd79-7dd1-4836-8a50-cecd856be150","Type":"ContainerDied","Data":"db1bce326d5437f7ead489c2e56becef3aa8c3c197f3754a3e611ae4b8e0d4a8"}
Oct 30 20:02:25.892999 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:25.892669    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv" event={"ID":"90a8fd79-7dd1-4836-8a50-cecd856be150","Type":"ContainerStarted","Data":"b2a4f907d6b7a0ad45082ad8da6ecf063df5fe1ba92e741ac21c12ed832028a0"}
Oct 30 20:02:27.036631 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.036602    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:27.142264 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.142229    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90a8fd79-7dd1-4836-8a50-cecd856be150-kube-api-access-hr2jq\" (UniqueName: \"kubernetes.io/projected/90a8fd79-7dd1-4836-8a50-cecd856be150-kube-api-access-hr2jq\") pod \"90a8fd79-7dd1-4836-8a50-cecd856be150\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") "
Oct 30 20:02:27.142464 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.142272    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-openshift-ca\") pod \"90a8fd79-7dd1-4836-8a50-cecd856be150\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") "
Oct 30 20:02:27.142464 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.142306    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-xks-ca\") pod \"90a8fd79-7dd1-4836-8a50-cecd856be150\" (UID: \"90a8fd79-7dd1-4836-8a50-cecd856be150\") "
Oct 30 20:02:27.142675 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.142649    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-openshift-ca" pod "90a8fd79-7dd1-4836-8a50-cecd856be150" (UID: "90a8fd79-7dd1-4836-8a50-cecd856be150"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:27.144372 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.144350    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90a8fd79-7dd1-4836-8a50-cecd856be150-kube-api-access-hr2jq" pod "90a8fd79-7dd1-4836-8a50-cecd856be150" (UID: "90a8fd79-7dd1-4836-8a50-cecd856be150"). InnerVolumeSpecName "kube-api-access-hr2jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:27.144477 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.144462    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-xks-ca" pod "90a8fd79-7dd1-4836-8a50-cecd856be150" (UID: "90a8fd79-7dd1-4836-8a50-cecd856be150"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:27.243680 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.243647    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr2jq\" (UniqueName: \"kubernetes.io/projected/90a8fd79-7dd1-4836-8a50-cecd856be150-kube-api-access-hr2jq\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:27.243680 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.243678    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-openshift-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:27.243911 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.243692    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90a8fd79-7dd1-4836-8a50-cecd856be150-maas-api-xks-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:27.905889 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.905857    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"
Oct 30 20:02:27.905889 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.905866    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv" event={"ID":"90a8fd79-7dd1-4836-8a50-cecd856be150","Type":"ContainerDied","Data":"b2a4f907d6b7a0ad45082ad8da6ecf063df5fe1ba92e741ac21c12ed832028a0"}
Oct 30 20:02:27.906129 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:27.905912    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2a4f907d6b7a0ad45082ad8da6ecf063df5fe1ba92e741ac21c12ed832028a0"
Oct 30 20:02:35.654158 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:35.654123    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"]
Oct 30 20:02:35.754298 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:35.754261    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"]
Oct 30 20:02:38.507931 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.507897    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"]
Oct 30 20:02:38.508333 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.508276    5232 cpu_manager.go:414] "containerMap: removing container" podUID="90a8fd79-7dd1-4836-8a50-cecd856be150" containerName="revoke-keys"
Oct 30 20:02:38.508333 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.508287    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90a8fd79-7dd1-4836-8a50-cecd856be150" containerName="revoke-keys"
Oct 30 20:02:38.508333 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.508297    5232 cpu_manager.go:414] "containerMap: removing container" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:38.508333 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.508302    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1d07fdd-e6f4-40e0-bea5-3e05fa5d2a32" containerName="cleanup"
Oct 30 20:02:38.508490 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.508381    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="90a8fd79-7dd1-4836-8a50-cecd856be150" containerName="revoke-keys"
Oct 30 20:02:38.511614 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.511592    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.547081 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.547048    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ggbq\" (UniqueName: \"kubernetes.io/projected/83879c32-17f9-4289-9a80-2b3c43fb4938-kube-api-access-9ggbq\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.547234 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.547103    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-xks-ca\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.547279 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.547241    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.562766 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.562731    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:02:38.610434 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.610402    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 20:02:38.613557 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.613520    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"]
Oct 30 20:02:38.647849 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.647796    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.648034 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.647877    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ggbq\" (UniqueName: \"kubernetes.io/projected/83879c32-17f9-4289-9a80-2b3c43fb4938-kube-api-access-9ggbq\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.648034 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.647942    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-xks-ca\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.648462 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.648439    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.650365 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.650342    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-xks-ca\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.707401 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.707370    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ggbq\" (UniqueName: \"kubernetes.io/projected/83879c32-17f9-4289-9a80-2b3c43fb4938-kube-api-access-9ggbq\") pod \"maas-api-revoke-sub-281e2be4-8l6ql\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:38.824396 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:38.824320    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:39.195138 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:39.195100    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"]
Oct 30 20:02:39.200721 ip-10-0-143-248 kubenswrapper[5232]: W1030 20:02:39.200688    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83879c32_17f9_4289_9a80_2b3c43fb4938.slice/crio-9a7b86a697d68b212d76b4253e6d32c7cc9bc946d04766505c0fc92457bd3a33 WatchSource:0}: Error finding container 9a7b86a697d68b212d76b4253e6d32c7cc9bc946d04766505c0fc92457bd3a33: Status 404 returned error can't find the container with id 9a7b86a697d68b212d76b4253e6d32c7cc9bc946d04766505c0fc92457bd3a33
Oct 30 20:02:39.970365 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:39.970326    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql" event={"ID":"83879c32-17f9-4289-9a80-2b3c43fb4938","Type":"ContainerStarted","Data":"ed2bbee1c048de05936163766008fe5b10b689673b9322f67d9d3dc0423f684b"}
Oct 30 20:02:39.970787 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:39.970399    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql" event={"ID":"83879c32-17f9-4289-9a80-2b3c43fb4938","Type":"ContainerStarted","Data":"9a7b86a697d68b212d76b4253e6d32c7cc9bc946d04766505c0fc92457bd3a33"}
Oct 30 20:02:40.082933 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:40.082884    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql" podStartSLOduration=2.082870912 podStartE2EDuration="2.082870912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:02:40.082162679 +0000 UTC m=+4552.666645675" watchObservedRunningTime="2026-10-30 20:02:40.082870912 +0000 UTC m=+4552.667353908"
Oct 30 20:02:40.655118 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:40.655073    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564" podUID="58c132fb-563a-4f6f-ae7c-2a6074100f8b" containerName="payload-processing" containerID="cri-o://1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e" gracePeriod=25
Oct 30 20:02:40.755204 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:40.755160    5232 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm" podUID="69a2fe95-4f24-4210-bf32-7791b2de59b4" containerName="payload-pre-processing" containerID="cri-o://0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5" gracePeriod=25
Oct 30 20:02:40.976877 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:40.976772    5232 generic.go:356] "Generic (PLEG): container finished" podID="83879c32-17f9-4289-9a80-2b3c43fb4938" containerID="ed2bbee1c048de05936163766008fe5b10b689673b9322f67d9d3dc0423f684b" exitCode=0
Oct 30 20:02:40.976877 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:40.976856    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql" event={"ID":"83879c32-17f9-4289-9a80-2b3c43fb4938","Type":"ContainerDied","Data":"ed2bbee1c048de05936163766008fe5b10b689673b9322f67d9d3dc0423f684b"}
Oct 30 20:02:42.125911 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.125883    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:42.185172 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.185137    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83879c32-17f9-4289-9a80-2b3c43fb4938-kube-api-access-9ggbq\" (UniqueName: \"kubernetes.io/projected/83879c32-17f9-4289-9a80-2b3c43fb4938-kube-api-access-9ggbq\") pod \"83879c32-17f9-4289-9a80-2b3c43fb4938\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") "
Oct 30 20:02:42.185367 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.185228    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-openshift-ca\") pod \"83879c32-17f9-4289-9a80-2b3c43fb4938\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") "
Oct 30 20:02:42.185367 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.185273    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-xks-ca\") pod \"83879c32-17f9-4289-9a80-2b3c43fb4938\" (UID: \"83879c32-17f9-4289-9a80-2b3c43fb4938\") "
Oct 30 20:02:42.185651 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.185628    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-openshift-ca" pod "83879c32-17f9-4289-9a80-2b3c43fb4938" (UID: "83879c32-17f9-4289-9a80-2b3c43fb4938"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:42.187362 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.187332    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83879c32-17f9-4289-9a80-2b3c43fb4938-kube-api-access-9ggbq" pod "83879c32-17f9-4289-9a80-2b3c43fb4938" (UID: "83879c32-17f9-4289-9a80-2b3c43fb4938"). InnerVolumeSpecName "kube-api-access-9ggbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:42.187362 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.187341    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-xks-ca" pod "83879c32-17f9-4289-9a80-2b3c43fb4938" (UID: "83879c32-17f9-4289-9a80-2b3c43fb4938"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:42.286777 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.286689    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ggbq\" (UniqueName: \"kubernetes.io/projected/83879c32-17f9-4289-9a80-2b3c43fb4938-kube-api-access-9ggbq\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:42.286777 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.286724    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-openshift-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:42.286777 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.286737    5232 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/83879c32-17f9-4289-9a80-2b3c43fb4938-maas-api-xks-ca\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:42.992769 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.992736    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql"
Oct 30 20:02:42.992994 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.992772    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-281e2be4-8l6ql" event={"ID":"83879c32-17f9-4289-9a80-2b3c43fb4938","Type":"ContainerDied","Data":"9a7b86a697d68b212d76b4253e6d32c7cc9bc946d04766505c0fc92457bd3a33"}
Oct 30 20:02:42.992994 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:42.992867    5232 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a7b86a697d68b212d76b4253e6d32c7cc9bc946d04766505c0fc92457bd3a33"
Oct 30 20:02:45.357338 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:45.357302    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm" podUID="69a2fe95-4f24-4210-bf32-7791b2de59b4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:02:45.357753 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:45.357299    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564" podUID="58c132fb-563a-4f6f-ae7c-2a6074100f8b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:02:46.187581 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:02:46.187540    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:02:55.356473 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:55.356432    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm" podUID="69a2fe95-4f24-4210-bf32-7791b2de59b4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:02:55.357310 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:55.356435    5232 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564" podUID="58c132fb-563a-4f6f-ae7c-2a6074100f8b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:02:55.888758 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:55.888720    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"]
Oct 30 20:02:55.936427 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:55.936384    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-1323a7-6b2cv"]
Oct 30 20:02:57.538626 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:02:57.538582    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90a8fd79-7dd1-4836-8a50-cecd856be150" path="/var/lib/kubelet/pods/90a8fd79-7dd1-4836-8a50-cecd856be150/volumes"
Oct 30 20:03:00.906952 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:00.906923    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 20:03:01.002337 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.002310    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 20:03:01.071414 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.071378    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58c132fb-563a-4f6f-ae7c-2a6074100f8b-kube-api-access-hhw65\" (UniqueName: \"kubernetes.io/projected/58c132fb-563a-4f6f-ae7c-2a6074100f8b-kube-api-access-hhw65\") pod \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\" (UID: \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\") "
Oct 30 20:03:01.071595 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.071436    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58c132fb-563a-4f6f-ae7c-2a6074100f8b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58c132fb-563a-4f6f-ae7c-2a6074100f8b-plugins-config-volume\") pod \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\" (UID: \"58c132fb-563a-4f6f-ae7c-2a6074100f8b\") "
Oct 30 20:03:01.071831 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.071789    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58c132fb-563a-4f6f-ae7c-2a6074100f8b-plugins-config-volume" pod "58c132fb-563a-4f6f-ae7c-2a6074100f8b" (UID: "58c132fb-563a-4f6f-ae7c-2a6074100f8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:03:01.073511 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.073491    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58c132fb-563a-4f6f-ae7c-2a6074100f8b-kube-api-access-hhw65" pod "58c132fb-563a-4f6f-ae7c-2a6074100f8b" (UID: "58c132fb-563a-4f6f-ae7c-2a6074100f8b"). InnerVolumeSpecName "kube-api-access-hhw65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:03:01.092949 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.092920    5232 generic.go:356] "Generic (PLEG): container finished" podID="58c132fb-563a-4f6f-ae7c-2a6074100f8b" containerID="1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e" exitCode=0
Oct 30 20:03:01.093121 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.092969    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"
Oct 30 20:03:01.093121 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.093002    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564" event={"ID":"58c132fb-563a-4f6f-ae7c-2a6074100f8b","Type":"ContainerDied","Data":"1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e"}
Oct 30 20:03:01.093121 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.093095    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564" event={"ID":"58c132fb-563a-4f6f-ae7c-2a6074100f8b","Type":"ContainerDied","Data":"8667a91e34124f0758138724f2bf2e5c8ef7ebe629abfd7272591efd212177c0"}
Oct 30 20:03:01.093258 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.093124    5232 scope.go:122] "RemoveContainer" containerID="1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e"
Oct 30 20:03:01.094340 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.094319    5232 generic.go:356] "Generic (PLEG): container finished" podID="69a2fe95-4f24-4210-bf32-7791b2de59b4" containerID="0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5" exitCode=0
Oct 30 20:03:01.094432 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.094368    5232 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"
Oct 30 20:03:01.094488 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.094366    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm" event={"ID":"69a2fe95-4f24-4210-bf32-7791b2de59b4","Type":"ContainerDied","Data":"0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5"}
Oct 30 20:03:01.094549 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.094507    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm" event={"ID":"69a2fe95-4f24-4210-bf32-7791b2de59b4","Type":"ContainerDied","Data":"7939ce77a7194a03a35627cd9f7e12212f224d01b421b58493d8ef627d45fc9e"}
Oct 30 20:03:01.102907 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.102889    5232 scope.go:122] "RemoveContainer" containerID="1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e"
Oct 30 20:03:01.103197 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:03:01.103177    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e\": container with ID starting with 1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e not found: ID does not exist" containerID="1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e"
Oct 30 20:03:01.103328 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.103207    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e"} err="failed to get container status \"1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e\": rpc error: code = NotFound desc = could not find container \"1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e\": container with ID starting with 1c3eeffe75e4782a07c8bfe13a0c51069e1e5c66bcb44c5bbca1cab3f92dfa8e not found: ID does not exist"
Oct 30 20:03:01.103328 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.103233    5232 scope.go:122] "RemoveContainer" containerID="0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5"
Oct 30 20:03:01.112035 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.112014    5232 scope.go:122] "RemoveContainer" containerID="0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5"
Oct 30 20:03:01.112329 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:03:01.112308    5232 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5\": container with ID starting with 0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5 not found: ID does not exist" containerID="0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5"
Oct 30 20:03:01.112382 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.112337    5232 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5"} err="failed to get container status \"0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5\": rpc error: code = NotFound desc = could not find container \"0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5\": container with ID starting with 0b11154b3365a72986c5e28e01fe2ed21038d66405fa133acb8e7bc03ba8cef5 not found: ID does not exist"
Oct 30 20:03:01.172097 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.172065    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69a2fe95-4f24-4210-bf32-7791b2de59b4-kube-api-access-xpl8f\" (UniqueName: \"kubernetes.io/projected/69a2fe95-4f24-4210-bf32-7791b2de59b4-kube-api-access-xpl8f\") pod \"69a2fe95-4f24-4210-bf32-7791b2de59b4\" (UID: \"69a2fe95-4f24-4210-bf32-7791b2de59b4\") "
Oct 30 20:03:01.172275 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.172184    5232 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69a2fe95-4f24-4210-bf32-7791b2de59b4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a2fe95-4f24-4210-bf32-7791b2de59b4-plugins-config-volume\") pod \"69a2fe95-4f24-4210-bf32-7791b2de59b4\" (UID: \"69a2fe95-4f24-4210-bf32-7791b2de59b4\") "
Oct 30 20:03:01.172387 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.172373    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhw65\" (UniqueName: \"kubernetes.io/projected/58c132fb-563a-4f6f-ae7c-2a6074100f8b-kube-api-access-hhw65\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:01.172430 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.172389    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58c132fb-563a-4f6f-ae7c-2a6074100f8b-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:01.172545 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.172522    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a2fe95-4f24-4210-bf32-7791b2de59b4-plugins-config-volume" pod "69a2fe95-4f24-4210-bf32-7791b2de59b4" (UID: "69a2fe95-4f24-4210-bf32-7791b2de59b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:03:01.174280 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.174248    5232 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a2fe95-4f24-4210-bf32-7791b2de59b4-kube-api-access-xpl8f" pod "69a2fe95-4f24-4210-bf32-7791b2de59b4" (UID: "69a2fe95-4f24-4210-bf32-7791b2de59b4"). InnerVolumeSpecName "kube-api-access-xpl8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:03:01.273634 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.273545    5232 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpl8f\" (UniqueName: \"kubernetes.io/projected/69a2fe95-4f24-4210-bf32-7791b2de59b4-kube-api-access-xpl8f\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:01.273634 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.273576    5232 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69a2fe95-4f24-4210-bf32-7791b2de59b4-plugins-config-volume\") on node \"ip-10-0-143-248.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:01.300468 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.300426    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"]
Oct 30 20:03:01.304379 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.304353    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1323a7-7b4454d49b-qs564"]
Oct 30 20:03:01.509938 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.509889    5232 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"]
Oct 30 20:03:01.514164 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.514134    5232 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1323a7-5fbf78db8c-bpxgm"]
Oct 30 20:03:01.538822 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.538730    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58c132fb-563a-4f6f-ae7c-2a6074100f8b" path="/var/lib/kubelet/pods/58c132fb-563a-4f6f-ae7c-2a6074100f8b/volumes"
Oct 30 20:03:01.539150 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:01.539134    5232 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69a2fe95-4f24-4210-bf32-7791b2de59b4" path="/var/lib/kubelet/pods/69a2fe95-4f24-4210-bf32-7791b2de59b4/volumes"
Oct 30 20:03:04.987335 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:04.987302    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dcb89d9-lwzcs_1a8f6a91-c365-409e-9eea-902eedd65d96/authorino/0.log"
Oct 30 20:03:05.387966 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:05.387930    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-jd8zk_a6d946b1-cb85-4bf5-8144-3a121af0b5d2/manager/0.log"
Oct 30 20:03:06.060922 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:06.060891    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-2kfmd_1fa31cdb-42cf-4f45-a06a-aab92e1f0558/kuadrant-console-plugin/0.log"
Oct 30 20:03:08.927603 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:08.927573    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-6fh88_d459d5ca-7637-41a0-845a-311991240ae9/manager/0.log"
Oct 30 20:03:09.179658 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:09.179586    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-5t7c4_0b3f10fb-bb7e-464a-be03-8553bf46f443/manager/0.log"
Oct 30 20:03:10.020294 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:10.020256    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/1.log"
Oct 30 20:03:11.303760 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:11.303725    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/0.log"
Oct 30 20:03:11.907766 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:11.907737    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dcb89d9-lwzcs_1a8f6a91-c365-409e-9eea-902eedd65d96/authorino/0.log"
Oct 30 20:03:16.186371 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:03:16.186336    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:03:22.981172 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:22.981120    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-6fh88_d459d5ca-7637-41a0-845a-311991240ae9/manager/0.log"
Oct 30 20:03:23.275040 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:23.274956    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-5t7c4_0b3f10fb-bb7e-464a-be03-8553bf46f443/manager/0.log"
Oct 30 20:03:24.168630 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:24.168590    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/1.log"
Oct 30 20:03:25.438624 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:25.438587    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/0.log"
Oct 30 20:03:26.610715 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:26.610632    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-281e2be4-8l6ql_83879c32-17f9-4289-9a80-2b3c43fb4938/revoke-keys/0.log"
Oct 30 20:03:28.953216 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:28.953182    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-wbfgw_5a4394b5-1f92-42de-85c6-b237231ba477/postgres/0.log"
Oct 30 20:03:30.468580 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:30.468545    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dcb89d9-lwzcs_1a8f6a91-c365-409e-9eea-902eedd65d96/authorino/0.log"
Oct 30 20:03:30.902689 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:30.902659    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-jd8zk_a6d946b1-cb85-4bf5-8144-3a121af0b5d2/manager/0.log"
Oct 30 20:03:31.361800 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:31.361771    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-2kfmd_1fa31cdb-42cf-4f45-a06a-aab92e1f0558/kuadrant-console-plugin/0.log"
Oct 30 20:03:33.686285 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:33.686250    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67b5c995fc6lch7_b17c47fb-fc48-4456-a64f-06ec2d144842/istio-proxy/0.log"
Oct 30 20:03:34.754771 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:34.754740    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-gshcc_ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a/istio-proxy/1.log"
Oct 30 20:03:35.270210 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:35.270173    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-v65bt_f035666f-23c9-494a-9963-6795a9f87e59/payload-pre-processing/0.log"
Oct 30 20:03:35.516037 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:35.516006    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-9zk2f_528babed-84aa-491f-85a9-16cc40b8edc9/payload-processing/0.log"
Oct 30 20:03:36.240084 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:36.240047    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-gshcc_ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a/istio-proxy/0.log"
Oct 30 20:03:36.840698 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:36.840665    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-8gxc4_3a777aa3-f14b-4ae3-b229-1b80b9ec54eb/main/0.log"
Oct 30 20:03:37.372036 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:37.372000    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-wtpr6_4c31cd17-e19d-43a0-9a77-f6819eae43fb/main/0.log"
Oct 30 20:03:37.922803 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:37.922766    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr9lx_e6d634df-343d-47bd-b07c-9d180268a5c7/main/0.log"
Oct 30 20:03:38.493269 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:38.493238    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-dlxsr_382fe2e8-957d-4996-a946-aeb825d0d31c/main/0.log"
Oct 30 20:03:46.186633 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:03:46.186595    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:03:49.779570 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.779489    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x98bk/must-gather-cns9q"]
Oct 30 20:03:49.779970 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.779923    5232 cpu_manager.go:414] "containerMap: removing container" podUID="69a2fe95-4f24-4210-bf32-7791b2de59b4" containerName="payload-pre-processing"
Oct 30 20:03:49.779970 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.779938    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69a2fe95-4f24-4210-bf32-7791b2de59b4" containerName="payload-pre-processing"
Oct 30 20:03:49.779970 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.779958    5232 cpu_manager.go:414] "containerMap: removing container" podUID="83879c32-17f9-4289-9a80-2b3c43fb4938" containerName="revoke-keys"
Oct 30 20:03:49.779970 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.779965    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83879c32-17f9-4289-9a80-2b3c43fb4938" containerName="revoke-keys"
Oct 30 20:03:49.780169 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.779982    5232 cpu_manager.go:414] "containerMap: removing container" podUID="58c132fb-563a-4f6f-ae7c-2a6074100f8b" containerName="payload-processing"
Oct 30 20:03:49.780169 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.779990    5232 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58c132fb-563a-4f6f-ae7c-2a6074100f8b" containerName="payload-processing"
Oct 30 20:03:49.780169 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.780094    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="69a2fe95-4f24-4210-bf32-7791b2de59b4" containerName="payload-pre-processing"
Oct 30 20:03:49.780169 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.780103    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="83879c32-17f9-4289-9a80-2b3c43fb4938" containerName="revoke-keys"
Oct 30 20:03:49.780169 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.780112    5232 memory_manager.go:365] "RemoveStaleState removing state" podUID="58c132fb-563a-4f6f-ae7c-2a6074100f8b" containerName="payload-processing"
Oct 30 20:03:49.785146 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.785123    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x98bk/must-gather-cns9q"
Oct 30 20:03:49.841334 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.841304    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x98bk\"/\"openshift-service-ca.crt\""
Oct 30 20:03:49.888799 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.888761    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x98bk\"/\"kube-root-ca.crt\""
Oct 30 20:03:49.889016 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.888943    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x98bk\"/\"default-dockercfg-h82gj\""
Oct 30 20:03:49.891128 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.891103    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x98bk/must-gather-cns9q"]
Oct 30 20:03:49.901425 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.901407    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9mbx\" (UniqueName: \"kubernetes.io/projected/3150bd02-ec11-4279-b9b0-641d56909171-kube-api-access-h9mbx\") pod \"must-gather-cns9q\" (UID: \"3150bd02-ec11-4279-b9b0-641d56909171\") " pod="openshift-must-gather-x98bk/must-gather-cns9q"
Oct 30 20:03:49.901526 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:49.901435    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3150bd02-ec11-4279-b9b0-641d56909171-must-gather-output\") pod \"must-gather-cns9q\" (UID: \"3150bd02-ec11-4279-b9b0-641d56909171\") " pod="openshift-must-gather-x98bk/must-gather-cns9q"
Oct 30 20:03:50.003047 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:50.003012    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9mbx\" (UniqueName: \"kubernetes.io/projected/3150bd02-ec11-4279-b9b0-641d56909171-kube-api-access-h9mbx\") pod \"must-gather-cns9q\" (UID: \"3150bd02-ec11-4279-b9b0-641d56909171\") " pod="openshift-must-gather-x98bk/must-gather-cns9q"
Oct 30 20:03:50.003221 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:50.003054    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3150bd02-ec11-4279-b9b0-641d56909171-must-gather-output\") pod \"must-gather-cns9q\" (UID: \"3150bd02-ec11-4279-b9b0-641d56909171\") " pod="openshift-must-gather-x98bk/must-gather-cns9q"
Oct 30 20:03:50.003382 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:50.003366    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3150bd02-ec11-4279-b9b0-641d56909171-must-gather-output\") pod \"must-gather-cns9q\" (UID: \"3150bd02-ec11-4279-b9b0-641d56909171\") " pod="openshift-must-gather-x98bk/must-gather-cns9q"
Oct 30 20:03:50.075625 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:50.075552    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9mbx\" (UniqueName: \"kubernetes.io/projected/3150bd02-ec11-4279-b9b0-641d56909171-kube-api-access-h9mbx\") pod \"must-gather-cns9q\" (UID: \"3150bd02-ec11-4279-b9b0-641d56909171\") " pod="openshift-must-gather-x98bk/must-gather-cns9q"
Oct 30 20:03:50.099479 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:50.099452    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x98bk/must-gather-cns9q"
Oct 30 20:03:50.511679 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:50.511620    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x98bk/must-gather-cns9q"]
Oct 30 20:03:50.516798 ip-10-0-143-248 kubenswrapper[5232]: W1030 20:03:50.516771    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3150bd02_ec11_4279_b9b0_641d56909171.slice/crio-a76936ae2d7860435f24b20313b6f1e3919d0fde67cd268589122ccfaacc4e31 WatchSource:0}: Error finding container a76936ae2d7860435f24b20313b6f1e3919d0fde67cd268589122ccfaacc4e31: Status 404 returned error can't find the container with id a76936ae2d7860435f24b20313b6f1e3919d0fde67cd268589122ccfaacc4e31
Oct 30 20:03:50.518616 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:50.518596    5232 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:03:51.363220 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:51.363182    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x98bk/must-gather-cns9q" event={"ID":"3150bd02-ec11-4279-b9b0-641d56909171","Type":"ContainerStarted","Data":"a76936ae2d7860435f24b20313b6f1e3919d0fde67cd268589122ccfaacc4e31"}
Oct 30 20:03:58.478711 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:03:58.478672    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kml2f_6e3b2c8e-e1e8-499f-9c8b-164e47a57f54/global-pull-secret-syncer/0.log"
Oct 30 20:04:01.435755 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:01.435718    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x98bk/must-gather-cns9q" event={"ID":"3150bd02-ec11-4279-b9b0-641d56909171","Type":"ContainerStarted","Data":"ceb05dc7698cc417a2acea944fffccb09934ab3a45e9e7bf2363a0314d8962ca"}
Oct 30 20:04:01.436199 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:01.435771    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x98bk/must-gather-cns9q" event={"ID":"3150bd02-ec11-4279-b9b0-641d56909171","Type":"ContainerStarted","Data":"95a6804fcdad42ac7d67cfc8ec07ac4afae46fe5c519718af2a1c2fb4cd17fe3"}
Oct 30 20:04:01.564547 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:01.564478    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-x98bk/must-gather-cns9q" podStartSLOduration=2.415694674 podStartE2EDuration="12.564458722s" totalImagesPullingTime="10.148764048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:01.56270231 +0000 UTC m=+4634.147185306" watchObservedRunningTime="2026-10-30 20:04:01.564458722 +0000 UTC m=+4634.148941741"
Oct 30 20:04:02.714072 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:02.714018    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xhv7g_cb6b70f0-1c02-4f23-bb51-b5cac08fa4dd/konnectivity-agent/0.log"
Oct 30 20:04:03.950368 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:03.950317    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-248.us-west-2.compute.internal_d0bf5677d5ce41c447fb39800129b147/haproxy/0.log"
Oct 30 20:04:04.690954 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:04.690919    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-6fh88_d459d5ca-7637-41a0-845a-311991240ae9/manager/0.log"
Oct 30 20:04:05.034712 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:05.034642    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-5t7c4_0b3f10fb-bb7e-464a-be03-8553bf46f443/manager/0.log"
Oct 30 20:04:05.184036 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:05.184005    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-6fh88_d459d5ca-7637-41a0-845a-311991240ae9/manager/0.log"
Oct 30 20:04:05.459538 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:05.459492    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-5t7c4_0b3f10fb-bb7e-464a-be03-8553bf46f443/manager/0.log"
Oct 30 20:04:05.663650 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:05.663624    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/0.log"
Oct 30 20:04:05.893855 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:05.893779    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/1.log"
Oct 30 20:04:06.146690 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:06.146604    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/0.log"
Oct 30 20:04:06.514871 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:06.514779    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/1.log"
Oct 30 20:04:08.221097 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:08.221059    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dcb89d9-lwzcs_1a8f6a91-c365-409e-9eea-902eedd65d96/authorino/0.log"
Oct 30 20:04:08.676273 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:08.676243    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-jd8zk_a6d946b1-cb85-4bf5-8144-3a121af0b5d2/manager/0.log"
Oct 30 20:04:09.281336 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.281304    5232 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"]
Oct 30 20:04:09.286531 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.286511    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.339841 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.339798    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j68lw\"/\"openshift-service-ca.crt\""
Oct 30 20:04:09.379117 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.379083    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-lib-modules\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.379280 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.379148    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-sys\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.379328 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.379269    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spsq9\" (UniqueName: \"kubernetes.io/projected/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-kube-api-access-spsq9\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.379328 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.379312    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-proc\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.379436 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.379414    5232 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-podres\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.387269 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.387243    5232 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j68lw\"/\"kube-root-ca.crt\""
Oct 30 20:04:09.387382 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.387284    5232 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j68lw\"/\"default-dockercfg-plqkf\""
Oct 30 20:04:09.390904 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.390835    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"]
Oct 30 20:04:09.480589 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480556    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-sys\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.480782 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480635    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spsq9\" (UniqueName: \"kubernetes.io/projected/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-kube-api-access-spsq9\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.480782 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480670    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-proc\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.480782 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480680    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-sys\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.480782 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480735    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-proc\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.480782 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480757    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-podres\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.481098 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480788    5232 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-lib-modules\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.481098 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480901    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-lib-modules\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.481098 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.480926    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-podres\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.531423 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.531343    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-2kfmd_1fa31cdb-42cf-4f45-a06a-aab92e1f0558/kuadrant-console-plugin/0.log"
Oct 30 20:04:09.540406 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.540374    5232 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spsq9\" (UniqueName: \"kubernetes.io/projected/d8f6fd36-c7be-4ffa-8d74-3b26181fd512-kube-api-access-spsq9\") pod \"perf-node-gather-daemonset-qn2tc\" (UID: \"d8f6fd36-c7be-4ffa-8d74-3b26181fd512\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.598216 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.598184    5232 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:09.981152 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:09.981117    5232 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"]
Oct 30 20:04:09.986480 ip-10-0-143-248 kubenswrapper[5232]: W1030 20:04:09.986442    5232 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd8f6fd36_c7be_4ffa_8d74_3b26181fd512.slice/crio-4ed6bf41be6ec56bae6eb1a2d055201e35d83f12a492b115773a5696ead12bca WatchSource:0}: Error finding container 4ed6bf41be6ec56bae6eb1a2d055201e35d83f12a492b115773a5696ead12bca: Status 404 returned error can't find the container with id 4ed6bf41be6ec56bae6eb1a2d055201e35d83f12a492b115773a5696ead12bca
Oct 30 20:04:10.496899 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:10.496855    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc" event={"ID":"d8f6fd36-c7be-4ffa-8d74-3b26181fd512","Type":"ContainerStarted","Data":"1f8dd2d332c99920b94def552c31bafb7d407aa20269d255c321cca9859a5aa7"}
Oct 30 20:04:10.497334 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:10.496920    5232 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc" event={"ID":"d8f6fd36-c7be-4ffa-8d74-3b26181fd512","Type":"ContainerStarted","Data":"4ed6bf41be6ec56bae6eb1a2d055201e35d83f12a492b115773a5696ead12bca"}
Oct 30 20:04:10.497334 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:10.496949    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:10.617421 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:10.617370    5232 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc" podStartSLOduration=1.61735485 podStartE2EDuration="1.61735485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:10.61547954 +0000 UTC m=+4643.199962562" watchObservedRunningTime="2026-10-30 20:04:10.61735485 +0000 UTC m=+4643.201837890"
Oct 30 20:04:10.692033 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:10.691623    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-6fh88_d459d5ca-7637-41a0-845a-311991240ae9/manager/0.log"
Oct 30 20:04:10.972204 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:10.972169    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-5t7c4_0b3f10fb-bb7e-464a-be03-8553bf46f443/manager/0.log"
Oct 30 20:04:11.227560 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:11.227480    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-6fh88_d459d5ca-7637-41a0-845a-311991240ae9/manager/0.log"
Oct 30 20:04:11.524551 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:11.524420    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-5t7c4_0b3f10fb-bb7e-464a-be03-8553bf46f443/manager/0.log"
Oct 30 20:04:11.830548 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:11.830420    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/0.log"
Oct 30 20:04:12.093505 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:12.093468    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/1.log"
Oct 30 20:04:12.281652 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:12.281619    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/0.log"
Oct 30 20:04:12.584611 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:12.584532    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/1.log"
Oct 30 20:04:16.188240 ip-10-0-143-248 kubenswrapper[5232]: E1030 20:04:16.188195    5232 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zhb5w" containerName="extractor"
Oct 30 20:04:16.498067 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:16.497991    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-6fh88_d459d5ca-7637-41a0-845a-311991240ae9/manager/0.log"
Oct 30 20:04:16.513228 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:16.513200    5232 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-qn2tc"
Oct 30 20:04:16.760794 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:16.760673    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-5t7c4_0b3f10fb-bb7e-464a-be03-8553bf46f443/manager/0.log"
Oct 30 20:04:17.100936 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:17.100907    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-6fh88_d459d5ca-7637-41a0-845a-311991240ae9/manager/0.log"
Oct 30 20:04:17.372322 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:17.372208    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/0.log"
Oct 30 20:04:17.382516 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:17.382486    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-5t7c4_0b3f10fb-bb7e-464a-be03-8553bf46f443/manager/0.log"
Oct 30 20:04:17.693049 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:17.692951    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/1.log"
Oct 30 20:04:18.040096 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:18.039998    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/0.log"
Oct 30 20:04:18.322080 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:18.322002    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7447664ff8-lfrzx_9070de12-c761-46e4-a362-2365219f92eb/manager/1.log"
Oct 30 20:04:20.104092 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:20.104051    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jv7v_d22062d6-b0ae-4eed-acd4-f1142e650f41/node-exporter/0.log"
Oct 30 20:04:20.520377 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:20.520293    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jv7v_d22062d6-b0ae-4eed-acd4-f1142e650f41/kube-rbac-proxy/0.log"
Oct 30 20:04:20.928145 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:20.928116    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8jv7v_d22062d6-b0ae-4eed-acd4-f1142e650f41/init-textfile/0.log"
Oct 30 20:04:22.019315 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:22.019279    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67b5c995fc6lch7_b17c47fb-fc48-4456-a64f-06ec2d144842/istio-proxy/0.log"
Oct 30 20:04:22.921467 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:22.921428    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67b5c995fc6lch7_b17c47fb-fc48-4456-a64f-06ec2d144842/istio-proxy/0.log"
Oct 30 20:04:25.170453 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:25.170413    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-gshcc_ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a/istio-proxy/0.log"
Oct 30 20:04:25.173083 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:25.173061    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-gshcc_ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a/istio-proxy/1.log"
Oct 30 20:04:25.364083 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:25.364050    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-gshcc_ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a/istio-proxy/0.log"
Oct 30 20:04:25.366805 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:25.366779    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-gshcc_ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a/istio-proxy/1.log"
Oct 30 20:04:25.490193 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:25.490091    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-v65bt_f035666f-23c9-494a-9963-6795a9f87e59/payload-pre-processing/0.log"
Oct 30 20:04:25.692786 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:25.692749    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-9zk2f_528babed-84aa-491f-85a9-16cc40b8edc9/payload-processing/0.log"
Oct 30 20:04:25.701381 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:25.701358    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-v65bt_f035666f-23c9-494a-9963-6795a9f87e59/payload-pre-processing/0.log"
Oct 30 20:04:25.934576 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:25.934535    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-9zk2f_528babed-84aa-491f-85a9-16cc40b8edc9/payload-processing/0.log"
Oct 30 20:04:26.969315 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:26.969270    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/0.log"
Oct 30 20:04:26.971246 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:26.971219    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/prometheus/1.log"
Oct 30 20:04:27.283956 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:27.283868    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/config-reloader/0.log"
Oct 30 20:04:27.689271 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:27.689242    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/thanos-sidecar/0.log"
Oct 30 20:04:28.095401 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:28.095369    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/kube-rbac-proxy-web/0.log"
Oct 30 20:04:28.505498 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:28.505417    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/kube-rbac-proxy/0.log"
Oct 30 20:04:28.914693 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:28.914657    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/kube-rbac-proxy-thanos/0.log"
Oct 30 20:04:29.350142 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:29.350097    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3179b985-7263-403d-b791-666db397de1b/init-config-reloader/0.log"
Oct 30 20:04:29.776140 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:29.776060    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-mpn8v_b3c313c3-122e-43ce-9d68-106881e7b7a5/prometheus-operator/0.log"
Oct 30 20:04:30.176310 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:30.176279    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-mpn8v_b3c313c3-122e-43ce-9d68-106881e7b7a5/kube-rbac-proxy/0.log"
Oct 30 20:04:31.864509 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:31.864470    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dcb89d9-lwzcs_1a8f6a91-c365-409e-9eea-902eedd65d96/authorino/0.log"
Oct 30 20:04:31.901109 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:31.901074    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67b5c995fc6lch7_b17c47fb-fc48-4456-a64f-06ec2d144842/istio-proxy/0.log"
Oct 30 20:04:32.075246 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:32.075209    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-jd8zk_a6d946b1-cb85-4bf5-8144-3a121af0b5d2/manager/0.log"
Oct 30 20:04:32.343644 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:32.343604    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58dcb89d9-lwzcs_1a8f6a91-c365-409e-9eea-902eedd65d96/authorino/0.log"
Oct 30 20:04:32.585167 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:32.585136    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-jd8zk_a6d946b1-cb85-4bf5-8144-3a121af0b5d2/manager/0.log"
Oct 30 20:04:32.684286 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:32.684206    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-2kfmd_1fa31cdb-42cf-4f45-a06a-aab92e1f0558/kuadrant-console-plugin/0.log"
Oct 30 20:04:33.028939 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:33.028858    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-2kfmd_1fa31cdb-42cf-4f45-a06a-aab92e1f0558/kuadrant-console-plugin/0.log"
Oct 30 20:04:34.061541 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:34.061502    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-gshcc_ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a/istio-proxy/0.log"
Oct 30 20:04:34.064786 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:34.064759    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8b64f9ff9-gshcc_ddbcb069-1f08-4ba2-bc8c-14cc6f40d90a/istio-proxy/1.log"
Oct 30 20:04:34.420023 ip-10-0-143-248 kubenswrapper[5232]: I1030 20:04:34.419990    5232 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-v65bt_f035666f-23c9-494a-9963-6795a9f87e59/payload-pre-processing/0.log"